Corrections for restriction of range and attenuation in criterion-related validation studies.

The results of 1,253 Navy enlisted personnel on the Navy Basic Test Battery and the Navy Enlisted Advancement Exam were used as predictor and criterion, respectively. Groups were selected according to 5 selection ratios, .10 to .50. Results were then compared to the true validity. Except for the selection ratio of .10, all of the corrected validity coefficients slightly over-estimated the true validity. Triple and 2-step correction procedures yielded equivalent results. In all cases, the corrected coefficients better estimated the unrestricted true validity. (4 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Correction for range restriction: An expanded typology.

A common research problem is the estimation of the population correlation between x and y from an observed correlation rxy obtained from a sample that has been restricted because of some sample selection process. Methods of correcting sample correlations for range restriction in a limited set of conditions are well-known. An expanded classification scheme for range-restriction scenarios is developed that conceptualizes range-restriction scenarios from various combinations of the following facets: (a) the variable(s) on which selection occurs (x, y and/or a 3rd variable z), (b) whether unrestricted variances for the relevant variables are known, and (c) whether a 3rd variable, if involved, is measured or unmeasured. On the basis of these facets, the authors describe potential solutions for 11 different range-restriction scenarios and summarize research to date on these techniques. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Within-groups correlations and their correction for attenuation.

"Situations frequently occur in which an investigator obtains the correlations between the same two variables in several groups of subjects and wants to obtain some over-all estimate of the degree of correlation between the two variables." Several techniques for evaluating such statistical problems are presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Correction for variance restriction in point-biserial correlations.

A correction procedure is proposed for adjusting point-biserial correlations for attenuation produced by inopportune splits in the dichotomous variable. The correction procedure permits estimation of the point-biserial correlation that would have been seen had equal proportions been present. Monte Carlo simulation evidence is provided for the accuracy of the correction procedure. Also, an example is provided from the employee turnover literature to illustrate how the correction procedure may be used. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Corrections for range restriction: An empirical investigation of conditions resulting in conservative corrections.

Analyzed the findings of over 700 criterion-related validity studies concerning (a) the relationship between the magnitude of the standard deviation (SD) of the predictor and the magnitude of the predictive validity, (b) the effect of corrections for range restriction, assuming explicit selection was based solely on the single predictor, and (c) the effect of corrections for range restriction, assuming selection was based on an unknown 3rd variable that had plausible correlations with the predictor and the criterion. As expected, a strong positive relationship was found in (a). Assumption of explicit selection, as in (b), reduced but did not eliminate the positive relationship between the SD and the corrected predictive validity. This relationship was reduced by corrections, as in (c). It is concluded that the usual correction for range restriction is better than the uncorrected coefficient but is still apt to provide a conservative estimate. More frequent use of corrections is encouraged. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An analysis of interperson correlations among thirty psychotics.

"A matrix of interperson correlations among 30 subjects based on above-median scores on 66 psychological test measures was factor analyzed by Thurstone's complete centroid method." It was concluded that "among chronic psychotics there exists a 'psychometric type' with a better-than-chance probability of recovery. This type is not distinguished by age, sex, or duration of illness… but… is characterized by generally poor performance on intellectual tasks, long reaction times, and poor time estimation." 15 references. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Implications of direct and indirect range restriction for meta-analysis methods and findings.

Range restriction in most data sets is indirect, but the meta-analysis methods used to date have applied the correction for direct range restriction to data in which range restriction is indirect. The authors show that this results in substantial undercorrections for the effects of range restriction, and they present meta-analysis methods for making accurate corrections when range restriction is indirect. Applying these methods to a well-known large-sample empirical database, the authors estimate that previous meta-analyses have underestimated the correlation between general mental ability and job performance by about 25%, indicating that this is potentially an important methodological issue in meta-analysis in general. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The restriction of range problem and nonignorable selection processes.

We describe a general model for estimating test validity when criterion scores cannot be observed for all cases. Unlike the traditional restriction of range correction formula, the proposed procedure can yield accurate estimates in problems in which an unobservable variable, related to y, is the basis for selection. This type of selection process is referred to as being nonignorable. In the nonignorable case, the regression statistics computed in the selected group, as well as the correction formula estimate, will be biased. The potential advantages of the proposed procedure over the correction formula approach are demonstrated in terms of hypothetical data sets in which the selection process is nonignorable. The shortcomings of the proposed analysis and future research needs are also discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Corrected formulas for computing sample sizes under indirect range restriction.

Asserts that the study of P. R. Sackett and B. E. Wade (see record 1983-31752-001) added significantly to the understanding of the interaction of statistical power, reliability, predictor intercorrelation, and selection ratio in determining the sample sizes required in validation studies where indirect range restriction is assumed. It is argued that, although Sackett and Wade's conclusions were correct, their results were marred by computational problems. The present note provides correct formulas for determining sample sizes under indirect range restriction. (4 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A proposed technique for the treatment of restriction of range of selected validation.

Evaluates traditional correction formulas designed to adjust for the effects of restriction of range in validation studies and proposes a new technique that resolves an important limitation of these earlier approaches. The conventional formulas require that the restricted variable be completely measured, even though most selection contexts involve restriction of range on both measured and unmeasured variables. The authors propose the use of a recently developed econometric technique that can correct for restriction of range attributable to both measured and unmeasured variables. The procedure explicitly recognizes the 2nd equation implicit in any validation study: the sample selection process. Based on well-established distribution theory and the assumption that the error terms in the 2 equations are distributed bivariate normal, the technique enables the researcher to estimate the bias associated with using the restricted sample. The limitations of the procedure and its applicability to a variety of selection problems are also discussed. (24 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Correction of validity coefficients for direct restriction in range occasioned by univariate selection.

Frequently, in evaluating a selection instrument, criterion data are available only for those applicants selected. Validities computed on the selectees are reduced by restriction in range and do not reflect the actual utility of the predictors. K. Pearson's (1903) correction for direct restriction in range uses the restricted validity (computed in the selectee group) and the predictor standard deviations in the unrestricted and the restricted groups. Often, the applicant group predictor standard deviation is not available. However, the practitioner may know, or be able to estimate, the selection ratio. An equation and a table for estimating the unrestricted validity from the restricted validity and the selection ratio are provided. (5 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic procedure.

Using computer simulation, the authors assessed the accuracy of J. E. Hunter, F. L. Schmidt, and H. Le's (2006) procedure for correcting for indirect range restriction, the most common type of range restriction, in comparison with the conventional practice of applying the Thorndike Case II correction for direct range restriction. Hunter et al.'s procedure produced more accurate estimates of both the mean and standard deviation in meta-analysis than the conventional procedure. Even when its key assumption that the effect of selection on a 3rd variable is fully mediated by the independent variable was violated, Hunter et al.'s procedure was still relatively more accurate than the conventional procedure. When applied to data from a previously published meta-analysis, the new procedure yielded results that led to different substantive conclusions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sign changes when correcting for range restriction: A note on Pearson's and Lawley's selection formulas.

K. Pearson (1903) recognized that the correlation coefficient was subject to distortion when a sample was censored or preselected in some way. He proposed 3 univariate correction formulas for better estimates in these circumstances. These have become well known from the work of R. L. Thorndike (1949). D. N. Lawley (1943) proposed a general solution usually called the multivariate correction for range restriction. Both Pearson's and Lawley's corrections are discussed and examples are presented. Of particular interest are the opportunities for the corrected correlations to change sign as a result of the correction. Numerical examples are presented that show that correlations can change signs in the Pearson-Thorndike Case 3 and in Lawley's general solution. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A cautionary note on the effects of range restriction on predictor intercorrelations.

The purpose of this research report is to highlight a unique set of issues that arise when considering the effects of range restriction in the context of estimating predictor intercorrelations. Three approaches are used to illustrate the issue: simulation, a concrete applied example, and a reanalysis of a meta-analysis of ability-interview correlations. The general conclusion is that a predictor intercorrelation can differ dramatically from the population value when both predictors are used in a composite that is used operationally for selection. The compensatory nature of a composite means that low scorers on one predictor can only obtain high scores on the composite if they obtain very high scores on the other predictor; this phenomenon distorts the correlation between the predictors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sampling variance in the correlation coefficient under indirect range restriction: Implications for validity generalization.

The authors conducted Monte Carlo simulations to investigate whether indirect range restriction (IRR) on 2 variables X and Y increases the sampling error variability in the correlation coefficient between them. The manipulated parameters were (a) IRR on X and Y (i.e., direct restriction on a third variable Z), (b) population correlations ρxy, ρxz, and ρyz and (c) sample size. IRR increased the sampling error variance in rxy to values as high as 8.50% larger than the analytically derived expected values. Thus, in the presence of IRR, validity generalization users need to make theory-based decisions to ascertain whether the effects of IRR are artifactual or caused by situational-specific moderating effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An empirical analysis of the correlations between leadership status and participation rates reported in the literature.

Many studies of small group leadership have reported a positive relationship between leadership status and the rate of verbal participation. The leadership/participation relationship has been attributed to (a) the performance of task leadership behaviors, (b) task skill and knowledge, (c) social status characteristics, and (d) the presence of observers. Hypotheses based on these 4 explanations were generated and tested using 72 correlations reported in the literature as data points. The relationship of a large number of situational variables to the leadership/participation rate relationship was also examined. In a multiple regression analysis, 4 variables accounted for 63% of the variance in the leadership/participation rate correlations. The 1st 3 explanations were supported. They can be integrated if the moderating effect of task competence and social status on the performance of task leadership behavior is recognized. (48 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Job-specific applicant pools and national norms for cognitive ability tests: Implications for range restriction corrections in validation research.

Correcting validity coefficients for range restriction requires the applicant pool predictor standard deviation (SD). As this is frequently unknown, some researchers use national norm SDs as estimates of the applicant pool SD. To test the proposition that job-specific applicant pools are markedly more homogeneous than broad samples of applicants for many jobs, job-specific applicant pool SDs on the Wonderlic Personnel Test for 80 jobs were compared with a large multijob applicant sample. For jobs at other than the lowest level of complexity, job-specific applicant pool SDs average 10% lower than the broad norm group SDs. 90% of the job-specific applicant pool SDs lie within 20% of the norm group SD, suggesting that reducing a norm group SD by 20% provides a conservative estimate of the applicant pool SD for use in range restriction corrections for other than low-complexity jobs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)