Academic Performance, Career Potential, Creativity, and Job Performance: Can One Construct Predict Them All?

This meta-analysis addresses the question of whether 1 general cognitive ability measure developed for predicting academic performance is valid for predicting performance in both educational and work domains. The validity of the Miller Analogies Test (MAT; W. S. Miller, 1960) for predicting 18 academic and work-related criteria was examined. MAT correlations with other cognitive tests (e.g., Raven's Matrices [J. C. Raven, 1965]; Graduate Record Examinations) also were meta-analyzed. The results indicate that the abilities measured by the MAT are shared with other cognitive ability instruments and that these abilities are generalizably valid predictors of academic and vocational criteria, as well as evaluations of career potential and creativity. These findings contradict the notion that intelligence at work is wholly different from intelligence at school, extending the voluminous literature that supports the broad importance of general cognitive ability (g). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Updating the trainability tests literature on Black–White subgroup differences and reconsidering criterion-related validity.

A number of applied psychologists have suggested that trainability test Black–White ethnic group differences are low or relatively low (e.g., Siegel & Bergman, 1975), though data are scarce. Likewise, there are relatively few estimates of criterion-related validity for trainability tests predicting job performance (cf. Robertson & Downs, 1989). We review and clarify the existing (and limited) literature on Black–White group differences on trainability tests, provide new trainability test data from a recent video-based trainability exam, and present archival data about how trainability test scores relate to cognitive ability, Black–White differences, and job performance. Consistent with hypotheses, our results suggest large correlations of trainability tests with cognitive ability (e.g., .80) and larger standardized ethnic group differences than previously thought (ds of 0.86, 1.10, and 1.21 for 3 samples). Results also suggest that trainability tests have higher validity than previously thought. Overall, our analysis provides a substantial amount of data to update our understanding of the use of trainability tests in personnel selection. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Employment testing: Old theories and new research findings.

Claims that cognitive ability tests of the kind generally used in personnel selection are valid predictors of successful performance for jobs in all settings. This controversial stance is supported by analyses that recast findings of invalid tests as instances of Type I error. Ideally, if an employer has large enough samples, perfectly reliable tests, and an unrestricted range of ability in the applicant pool, the most widely used types of standardized tests should be valid in all job situations, and the notion of job-specific validity would no longer hold. The authors argue against previous reservations about the suitability of cognitive ability tests for employee selection that were made on the basis of their supposed limited applicability, their bias, and their ultimate contribution to workforce productivity. (56 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Homogeneity of test validity.

In the present investigation we examined the extent to which the validity of three cognitive ability tests varied across test score range with respect to predicting job performance. This study was a partial replication and extension of R. Lee and P. P. Foley (see record 1987-09206-001). The 24,219 participants in the current study were obtained from the General Aptitude Test Battery (GATB) data base. Measures included in the analyses were general cognitive ability, numerical ability, and verbal ability. Results did not confirm Lee and Foley's findings. Specifically, the validity of the tests used in this study did not appreciably vary at different points along the test score range. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing common sense.

Predictors of success in school, such as conventional psychometric intelligence (e.g., IQ) tests, are less predictive of success out of school. Even the most charitable estimates of the relation between intelligence test scores and real-world criteria such as job performance indicate that approximately three fourths of the variance in real-world performance is not accounted for by intelligence test performance. Researchers have begun to explore new constructs in search of measures to supplement existing cognitive ability tests as predictors of real-world performance. Among the most promising constructs is practical intelligence, or common sense. Performance on measures of practical intelligence predicts real-world criteria such as job performance but is relatively unrelated to performance on intelligence tests and other common selection measures. Consequently, its contribution to prediction is largely independent of the contributions of existing measures, including measures of cognitive ability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reconsidering vocational interests for personnel selection: The validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions.

Although vocational interests have a long history in vocational psychology, they have received extremely limited attention within the recent personnel selection literature. We reconsider some widely held beliefs concerning the (low) validity of interests for predicting criteria important to selection researchers, and we review theory and empirical evidence that challenge such beliefs. We then describe the development and validation of an interests-based selection measure. Results of a large validation study (N = 418) reveal that interests predicted a diverse set of criteria—including measures of job knowledge, job performance, and continuance intentions—with corrected, cross-validated Rs that ranged from .25 to .46 across the criteria (mean R = .31). Interests also provided incremental validity beyond measures of general cognitive aptitude and facets of the Big Five personality dimensions in relation to each criterion. Furthermore, with a couple exceptions, the interest scales were associated with small to medium subgroup differences, which in most cases favored women and racial minorities. Taken as a whole, these results appear to call into question the prevailing thought that vocational interests have limited usefulness for selection. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the use of personality measures in personnel selection.

Discusses the use of personality as a predictor in personnel selection. Recent data has suggested that personality measures (PRMs) are related to performance criteria that are unrelated to cognitive ability when the traits measured are conceptually related to these criteria. It seems that PRMs may predict job performance dimensions that cannot be predicted by cognitive ability measures. The use of PRMs in personnel selection may be warranted when a careful job analysis is undertaken to determine which performance dimensions may be related to personality traits. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Positive manifold limits the relevance of content-matching strategies for validating selection test batteries.

Inferences about the relationships between scores on selection tests and measures of job performance are often made on the basis of an assessment of the match between the content of the test and the content of the job. However, there is little evidence that these test-to-job comparisons have any bearing on the criterion-related validity of selection tests. The authors show that conclusions reached in analyses of cognitive tests—that content matching is largely irrelevant to criterion-related validity—can be generalized to most sets of selection tests (e.g., psychomotor and performance tests, interview ratings, biodata scores, knowledge tests, work sample tests) that are positively correlated with one another and with the criterion. When the universe of potential predictors shows positive manifold, almost all possible sets of test batteries will yield similar outcomes and show similar validities, regardless of whether the content of these tests matches the content of the job. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity generalization results for two job groups in the petroleum industry.

Estimated the transituational generalizability (or "transportability") of the validities of 4 types of cognitive tests (Bennett Mechanical Comprehension Test, Richardson-Bellows-Henry Chemical Comprehension and RBH Arithmetic Reasoning Tests, and general intelligence) and a weighted biographical information blank for performance in 2 petroleum industry job groups. Generalizability was strongly supported for mechanical and chemical comprehension tests for both jobs. In the case of the chemical comprehension tests, virtually all variance of observed validity coefficients was accounted for by artifacts, and thus the hypothesis of situational specificity was rejected. Support for generalizability was substantial for general mental ability and arithmetic reasoning tests. It was found, however, that corrections for variance due to sampling error accounted for an average of 90% of all variance due to artifacts, indicating the relative unimportance of differences between sudies in criterion reliability and in range restriction in accounting for variation in observed validities. Generalizable multivariate validities were estimated for various test batteries using beta and unit weights. Finally, true score beta weights were used to estimate the causal role of the 4 cognitive abilities in job performance. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Role of social desirability in personality testing for personnel selection: The red herring.

Response bias continues to be the most frequently cited criticism of personality testing for personnel selection. The authors meta-analyzed the social desirability literature, examining whether social desirability functions as a predictor for a variety of criteria, as a suppressor, or as a mediator. Social desirability scales were found not to predict school success, task performance, counterproductive behaviors, and job performance. Correlations with the Big Five personality dimensions, cognitive ability, and years of education are presented along with empirical evidence that (a) social desirability is not as pervasive a problem as has been anticipated by industrial-organizational psychologists, (b) social desirability is in fact related to real individual differences in emotional stability and conscientiousness, and (c) social desirability does not function as a predictor, as a practically useful suppressor, or as a mediator variable for the criterion of job performance. Removing the effects of social desirability from the Big Five dimensions of personality leaves the criterion-related validity of personality constructs for predicting job performance intact. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Quantifying the effects of psychological interventions on employee job performance and work-force productivity.

Quantifying the economic impact of psychological programs in organizations requires determination of (a) the size and variability of the resulting increase in job performance and (b) the economic value of the increase in job performance. New methods of meta-analysis allow attainment of the 1st of these; and in relation to the 2nd, utility analysis methods provide the ability to translate job performance increases into estimates of the economic value of the program. In the area of personnel selection, many meta-analytic studies have resulted in precise and generalizable estimates of the validity of cognitive ability tests and other selection procedures. Utility analyses show that the job performance increases resulting from use of valid selection methods have substantial economic value. Valid selection produces major increases in work-force productivity. It is concluded that the combined effects of selection and nonselection interventions can be expected to produce substantial increases in workforce productivity. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Incremental validity of situational judgment tests.

Using 3 different samples, the authors assessed the incremental validity of situational judgment inventories (SJIs), relative to job knowledge, cognitive ability, job experience, and conscientiousness, in the prediction of job performance. The SJI was a valid predictor in all 3 samples and incrementally so in 2 samples. Relative to the other predictors, SJI's partial correlation with performance, controlling for the other 4 predictors, was superior in most comparisons. Subgroup differences on the SJI also appear to be less than those for cognitive ability and job knowledge, but greater than differences in conscientiousness. The SJI should prove to be a valuable additional measure in the prediction of job performance, but several additional areas of research are suggested. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing the constancy of validity with computer-generated sampling distributions of the multiplicative model variance estimate: Results for petroleum industry validation research.

A comparison of computer-generated sampling distributions showed that increases in total N resulting from greater sample size/validity study reduced the sampling variation of multiplicative model estimates of true validity variance more than did increases in total N resulting from a larger number of studies. Validity coefficients, range restriction, and criterion reliability data for 5 tests predicting job performance and training performance were obtained from studies conducted at petroleum industry plants. Those tests included the Richardson-Bellows-Henry Ability, Arithmetic Reasoning, and Chemical Comprehension Tests; the Bennett Mechanical Comprehension Tests; and the Mathematics scale of the California Achievement Tests. Studies for 3 job categories (operation, maintenance, and laboratory) were combined for analysis to maximize sensitivity to job and plant differences. Sampling distributions of true validity variance estimates were generated for each test–criterion type combination. A comparison of actual variance estimates with critical values indicated that 3 of the 5 tests did not vary significantly in true validities for job performance or for training criteria. Results for 2 arithmetic reasoning tests support the existence of variation in true validities for job performance and for training criteria. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Importance of Job Autonomy, Cognitive Ability, and Job-Related Skill for Predicting Role Breadth and Job Performance.

Role theory suggests and empirical research has found that there is considerable variation in how broadly individuals define their jobs. We investigated the theoretically meaningful yet infrequently studied relationships between incumbent job autonomy, cognitive ability, job-related skill, role breadth, and job performance. Using multiple data sources and multiple measurement occasions in a field setting, we found that job autonomy, cognitive ability, and job-related skill were positively related to role breadth, accounting for 23% of the variance in role breadth. In addition, role breadth was positively related to job performance and was found to mediate the relationship between job autonomy, cognitive ability, job-related skill, and job performance. These results add to our understanding of the factors that predict role breadth, as well as having implications for how job aspects and individual characteristics are translated into performance outcomes and the treatment of variability in incumbent reports of job tasks. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fair employment, sound psychometric practice, and reality: A dilemma and a partial solution.

Considers that while many of the tests used in employee selection are valid nomothetically, they penalize the individual. The fairness of certain validity studies (e.g., the biodata approach and standardized tests of mental ability) is questioned. It is suggested that the use of job simulation tests may remedy this problem. It is noted that such tests would be more expensive and might not apply to certain jobs, but would allow the direct assessment of the behavior desired and permit the candidate to learn more about the position. It is proposed that the American Psychological Association publicly endorse the use of job simulations, consulting psychologists stress the advantages of job simulations and that business and educational institutions be encouraged to provide validity data for their specific situations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection.

In high-stakes selection among candidates with considerable domain-specific knowledge and experience, investigations of whether high-fidelity simulations (assessment centers; ACs) have incremental validity over low-fidelity simulations (situational judgment tests; SJTs) are lacking. Therefore, this article integrates research on the validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations in advanced-level high-stakes settings. A model and hypotheses of how these 3 predictors work in combination to predict job performance were developed. In a sample of 196 applicants, all 3 predictors were significantly related to job performance. Both the SJT and the AC had incremental validity over the knowledge test. Moreover, the AC had incremental validity over the SJT. Model tests showed that the SJT fully mediated the effects of declarative knowledge on job performance, whereas the AC partially mediated the effects of the SJT. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Validity and utility of alternative predictors of job performance.

Meta-analysis of the cumulative research on various predictors of job performance showed that for entry-level jobs there was no predictor with validity equal to that of ability, which had a mean validity of .53. For selection on the basis of current job performance, the work sample test, with mean validity of .54, was slightly better. For federal entry-level jobs, substitution of an alternative predictor would cost from $3.12 (job tryout) to $15.89 billion/year (age). Hiring on ability had a utility of $15.61 billion/year but affected minority groups adversely. Hiring on ability by quotas would decrease utility by 5%. A 3rd strategy—using a low cutoff score—would decrease utility by 83%. Using other predictors in conjunction with ability tests might improve validity and reduce adverse impact, but there is as yet no database for studying this possibility. (89 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of the predictive validity of tests of mental ability for classroom performance in elementary grades.

Examined the relationship between scores on tests of mental ability and classroom performance in approximately 5,200 British early elementary school children (2nd–4th graders). The predictive validity of tests of mental ability for classroom performance recently has come under question, especially when used to examine members of disadvantaged groups. The few relevant studies that have used grades as their criteria of performance have produced contradictory findings. The present investigation yielded a strong predictive relationship between mental ability scores and classroom performance, irrespective of Ss' socioeconomic status (SES) and, in addition, indicated that the predictions of academic performance derived from standardized tests are not biased against children of lower SES. These findings are used as the basis for discussing possible reasons for inconsistencies in the past research and to suggest the appropriate function of mental ability tests in elementary education. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)