Validity and fairness of physical ability tests for predicting performance in craft jobs.

Problems relating to performance, accidents, and turnover in outdoor telephone craft jobs stimulated 2 experiments aimed at developing and validating a physical test battery. Based on job analysis results, a battery of 9 measures was administered to a sample of 128 Ss (83 males and 45 females) in Exp I. A 2-test battery (dynamic arm strength and reaction time), valid for predicting job task performance and turnover, was selected. Regression equations for males and females were not significantly different. Exp II included a sample of 210 Ss (132 males and 78 females). A 3-test battery consisting of a body density measure, a balance test, and a static strength test was selected based on relationships with training performance. No significant differences were found in the regression equations for males compared to females. The Exp II battery was also significantly related to field performance, training completion, and accidents and was valid for the Exp I criteria. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection.

In high-stakes selection among candidates with considerable domain-specific knowledge and experience, investigations of whether high-fidelity simulations (assessment centers; ACs) have incremental validity over low-fidelity simulations (situational judgment tests; SJTs) are lacking. Therefore, this article integrates research on the validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations in advanced-level high-stakes settings. A model and hypotheses of how these 3 predictors work in combination to predict job performance were developed. In a sample of 196 applicants, all 3 predictors were significantly related to job performance. Both the SJT and the AC had incremental validity over the knowledge test. Moreover, the AC had incremental validity over the SJT. Model tests showed that the SJT fully mediated the effects of declarative knowledge on job performance, whereas the AC partially mediated the effects of the SJT. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Use of situational judgment tests to predict job performance: A clarification of the literature.

Although situational judgment tests have a long history in the psychological assessment literature and continue to be frequently used in employment contexts, there has been virtually no summarization of this literature. The purpose of this article is to review the history of such tests and present the results of a meta-analysis on criterion-related and construct validity. On the basis of 102 coefficients and 10,640 people, situational judgment tests showed useful levels of validity (p?=?.34) that were generalizable. A review of 79 correlations between situational judgment tests and general cognitive ability involving 16,984 people indicated that situational judgment tests typically evidence relationships with cognitive ability (p?=?.46). On the basis of the literature review and meta-analytic findings, implications for the continued use of situational judgment tests are discussed, particularly in terms of recent investigations into tacit knowledge. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Inferences from personnel tests and their validity": Correction to Lawshe.

Reports an error in "Inferences from personnel tests and their validity" by C. H. Lawshe (Journal of Applied Psychology, 1985[Feb], Vol 70[1], 237-238). On page 238, line 4, the word "each" appears and should be "such." The sentence will, therefore, refer "to the use of such cognitive processes as inductive and deductive reasoning and such characteristics of temperament as emotional stability and self-esteem." (The following abstract of the original article appeared in record 1985-16032-001.) Contends that despite clear definitions in standard sources, psychologists persistently refer to the validity of tests instead of the validity of inferences from test scores. This persistence leads to references to "kinds of validity" when, in fact, there are "kinds of validity analysis strategies" whereby data are collected or generated to determine or defend the extent, degree, or strength of the inference or inferences that can be made from a set of test scores. It is concluded that content validity analysis strategies are appropriate only when the job behavior under scrutiny falls at the observation end of the continuum; when such behavior approaches the abstract end of the continuum, a construct validity analysis strategy is indicated. (5 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Retaking ability tests in a selection setting: Implications for practice effects, training performance, and turnover.

This field study investigated the effect of retaking identical selection tests on subsequent test scores of 4,726 candidates for law enforcement positions. For both cognitive ability and oral communication ability selection tests, candidates produced significant score increases between the 1st and 2nd and the 2nd and 3rd test administrations. Furthermore, the repeat testing relationships with posthire training performance and turnover were examined in a sample of 1,515 candidates eventually selected into the organization. As predicted from persistence and continuance commitment rationales, the number of tests necessary to gain entry into the organization was positively associated with training performance and negatively associated with turnover probability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Traditional tests and job simulations: Minority and majority performance and test validities.

Performance of 565 minority and 414 majority job applicants on traditional paper-and-pencil tests and on a telephone simulation that measured similar constructs was assessed. Models of the measurement characteristics of these 2 test batteries indicated larger subgroup mean differences on the traditional tests than on the simulation. Correlations between traditional tests were lower in the majority sample than in the minority sample, and the variance of the minority candidates' scores on the traditional tests was much larger than the variance of majority applicants' scores on the same measures. The validity of the simulation was lower than the validity of the traditional tests. This study replicates previous laboratory research that has indicated smaller subgroup differences on simulations than on paper-and-pencil tests and extends this research by providing evidence of the relative validity of these 2 types of measures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Correction to Schmidt, Hunter, and Pearlman.

Reports an error in the original article by Schmidt et al. (Journal of Applied Psychology. Vol. 66(2) Apr 1981, 166-185.). On page 169, Table 2 has an error. The values of the mean observed validity and mean true validity for the memory test type in job family B should be .20 and .42, respectively (not .42 and .43). (The following abstract of this article originally appeared in record 1981-22525-001.) Two studies, with a total sample size of 400,000 Ss and with the US Department of Labor's Dictionary of Occupational Job Titles (1977), examined the traditional belief that between-job task differences cause aptitude tests to be valid for some jobs but not for others. Results indicate that aptitude tests are valid across jobs, since the moderating effect of tasks(a) is negligible even when jobs differ grossly in task makeup and (b) is probably nonexistent when task differences are less extreme. Findings have implications for validity generalization, the use of task-oriented job analysis in selection research, criterion construction, moderator research, and proper interpretation of the US's Uniform Guidelines on Employee Selection Procedures. It is concluded that the belief that tasks are important moderators of test validities can be traced to behaviorist assumptions introduced into personnel psychology in the early 1960's and that, in retrospect, these assumptions are false. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Racial/ethnic differences in the criterion-related validity of cognitive ability tests: A qualitative and quantitative review.

The correlation between cognitive ability test scores and performance was separately meta-analyzed for Asian, Black, Hispanic, and White racial/ethnic subgroups. Compared to the average White observed correlation ( = .33, N = 903,779), average correlations were lower for Black samples ( = .24, N = 112,194) and Hispanic samples ( = .30, N = 51,205) and approximately equal for Asian samples ( = .33, N = 80,705). Despite some moderating effects (e.g., type of performance criterion, decade of data collection, job complexity), validity favored White over Black and Hispanic test takers in almost all conditions that included a sizable number of studies. Black–White validity comparisons were possible both across and within the 3 broad domains that use cognitive ability tests for high-stakes selection and placement: civilian employment, educational admissions, and the military. The trend of lower Black validity was repeated in each domain; however, average Black–White validity differences were largest in military studies and smallest in educational and employment studies. Further investigation of the reasons for these validity differences is warranted. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach.

A review of research on job performance suggests 3 broad components: task, citizenship, and counterproductive performance. This study examined the relative importance of each component to ratings of overall performance by using an experimental policy-capturing design. Managers in 5 jobs read hypothetical profiles describing employees' task, citizenship, and counterproductive performance and provided global ratings of performance. Within-subjects regression analyses indicated that the weights given to the 3 performance components varied across raters. Hierarchical cluster analyses indicated that raters' policies could be grouped into 3 homogeneous clusters: (a) task performance weighted highest, (b) counterproductive performance weighted highest, and (c) equal and large weights given to task and counterproductive performance. Hierarchical linear modeling indicated that demographic variables were not related to raters' weights. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Men, family-role quality, job-role quality, and physical health": Correction to Barnett and Marshall.

Reports an error in "Men, family-role quality, job-role quality, and physical health" by Rosalind C. Barnett and Nancy L. Marshall (Health Psychology, 1993[Jan], Vol 12[1], 48-55). In this article the title for Table 1 (p. 50) should be "Correlations Between Rewards and Concerns in the Job, Marital, and Parent Roles." Also, the last item in the first column of the table should read "6. Parent-role concerns." (The following abstract of the original article appeared in record 1993-21527-001.) The relationship between the quality of family roles, as spouse and as parent, and the quality of men's job role, on the one hand, and their reports of physical symptoms, on the other, was examined in men drawn from a random sample of 300 dual-earner couples. The main effect of marital-role quality (i.e., rewards and concerns) on physical health was estimated, as was the Marital-Role Quality × Job-Role Quality interaction effect. With respect to the present role, the effect of both parent-role occupancy and parent-role quality was estimated. After controlling for job-role quality, it was found that parent-role occupancy had neither main nor interactive effects. However, among the men who occupied all 3 roles (n = 180), only parent-role concerns were a significant predictor of physical health reports. Thus, concerns in the parent role are a major unmeasured source of variance in studies of the stress-illness relationship in men. [An erratum concerning this article appears in Health Psychology, 1993(Mar), Vol 12(2), 92. Table 1 is corrected.] (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analytic investigation of conscientiousness in the prediction of job performance: Examining the intercorrelations and the incremental validity of narrow traits.

Researchers of broad and narrow traits have debated whether narrow traits are important to consider in the prediction of job performance. Because personality-performance relationship meta-analyses have focused almost exclusively on the Big Five, the predictive power of narrow traits has not been adequately examined. In this study, the authors address this question by meta-analytically examining the degree to which the narrow traits of conscientiousness predict above and beyond global conscientiousness. Results suggest that narrow traits do incrementally predict performance above and beyond global conscientiousness, yet the degree to which they contribute depends on the particular performance criterion and occupation in question. Overall, the results of this study suggest that there are benefits to considering the narrow traits of conscientiousness in the prediction of performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Review of Performance Norms on Job Applications: Reliability and Validity Statistics.

Reviews the book, Performance Norms on Job Applications: Reliability and Validity Statistics. This manual includes the performance norms and standardization of each of the forms of the Wonderlic Personnel Test. The sample comprises 251,253 job applicants. The test itself is a measure of a persons trainability. Over one thousand businesses, industries and government departments generated data for the study. Analysis for all forms of the test is undertaken for age, sex, educational level, job classification, region, etc. Canadian data is presented separately and further stratified by province and one city (Toronto). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using working memory theory to investigate the construct validity of multiple-choice reading comprehension tests such as the SAT.

When taking multiple-choice tests of reading comprehension such as the Scholastic Assessment Test (SAT), test takers use a range of strategies that vary in the extent to which they emphasize reading the questions versus reading the passages. Researchers have challenged the construct validity of these tests because test takers can achieve better-than-chance performance even if they do not read the passages at all. By using an individual-differences approach that compares the relative power of working memory span to predict SAT performance for different test-taking strategies, the authors show that the SAT appears to be tapping reading comprehension processes as long as test takers engage in at least some reading of the passages themselves. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of Graduate Record Examinations (GRE) General Test Scores for Admissions to Colleges of Veterinary Medicine.

From the standpoint of test validation, veterinary medicine provides both a unique context in which to study the validity of Graduate Record Examinations (GRE) test scores and a singular opportunity to address the shortcomings typical of many GRE validity studies. This article documents a study of the validity of the GRE General Test for predicting 1st-year grade averages in a comprehensive sample of veterinary medical colleges. For each of 16 veterinary medical colleges, statistical corrections were applied to correct for the effects of range restriction in the predictors and unreliability of the criterion. When fully corrected for both range restriction and unreliability, the resulting validity coefficients were, on average, .53 for the combination of all 3 GRE General Test scores, .59 for undergraduate grade point average, and .71 for GRE scores and undergraduate grade point average together. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of testing accommodations on students' performances and reactions to testing.

This study examined the effects of testing accommodations on students' test performances and reactions to the use of testing accommodations. Participants (N = 170) were fourth- and eighth-grade students, with and without disabilities. All students were administered, with and without accommodations, equivalent forms of widely used math and reading tests. Students completed a questionnaire to summarize their reactions to the use of accommodations. The findings indicated testing accommodations overall had a positive impact on students' individual reading and math scores. Furthermore, testing accommodations had a differential positive effect on reading scores for students with disabilities compared to students without disabilities. The relationship between students' perceptions of testing accommodations and the effects of testing accommodations on their test performances was not significant, although most students had positive perceptions of testing accommodations. Students perceived the provision of accommodations as fair for students without disabilities and more fair for students with disabilities. These findings are interpreted within a validity framework and contribute to a greater understanding of testing accommodations by integrating information concerning effects on scores and consequential effects on test-takers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Academic Performance, Career Potential, Creativity, and Job Performance: Can One Construct Predict Them All?

This meta-analysis addresses the question of whether 1 general cognitive ability measure developed for predicting academic performance is valid for predicting performance in both educational and work domains. The validity of the Miller Analogies Test (MAT; W. S. Miller, 1960) for predicting 18 academic and work-related criteria was examined. MAT correlations with other cognitive tests (e.g., Raven's Matrices [J. C. Raven, 1965]; Graduate Record Examinations) also were meta-analyzed. The results indicate that the abilities measured by the MAT are shared with other cognitive ability instruments and that these abilities are generalizably valid predictors of academic and vocational criteria, as well as evaluations of career potential and creativity. These findings contradict the notion that intelligence at work is wholly different from intelligence at school, extending the voluminous literature that supports the broad importance of general cognitive ability (g). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Video-based versus written situational judgment tests: A comparison in terms of predictive validity.

In this study, the authors examined whether video-based situational judgment tests (SJTs) have higher predictive validity than written SJTs (keeping verbal content constant). The samples consisted of 1,159 students who completed a video-based version of an SJT and 1,750 students who completed the same SJT in a written format. The study was conducted in a high stakes testing context. The video-based version of an interpersonally oriented SJT had a lower correlation with cognitive ability than did the written version. It also had higher predictive and incremental validity for predicting interpersonally oriented criteria than did the written version. In this high stakes context, applicants also reacted relatively favorably to the SJTs, although there was no significant difference in face validity between the formats. These findings suggest that SJT format changes be made with caution and that validation evidence is needed when changes are proposed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparative validity of two projective and two structured attitude tests in a prison population.

Which is better for assessing personality—structured or projective devices? "Attitude toward Home & Parents and Attitude toward Law & Justice of 79 prison inmates were each measured by a sentence completion test and a structured attitude test. As examined through a multitrait-multimethod matrix, these tests were found to validate each other quite satisfactorily. Insofar as the two measurement approaches differed at all in the efficacy with which they differentiated crime groups among the prisoners, the structured tests were slightly the better." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Disentangling the effects of voice: The incremental roles of opportunity, behavior, and instrumentality in predicting procedural fairness.

Many voice studies have failed to distinguish among voice opportunity, perceived voice opportunity, voice behavior, and voice instrumentality. Thus, the authors sought to clarify the roles of each in determining procedural fairness perceptions. Controlling for the effect of voice opportunity, each of the 3 remaining constructs was hypothesized to predict fairness. Furthermore, voice instrumentality was hypothesized to moderate the effect of voice behavior on fairness. Undergraduates (N=102; 81 for some analyses) participated in an orientation-week design simulation in which voice opportunity was manipulated. The results indicated significant incremental effects of perceived voice opportunity and the predicted Voice Instrumentality × Voice Behavior interaction. Fairness was lowest for individuals who were denied voice opportunity, perceived less voice opportunity, and provided high levels of noninstrumental voice behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"When your top choice turns you down: Effect of rejected offers on the utility of selection tests": Correction to Murphy.

Reports an error in "When your top choice turns you down: Effect of rejected offers on the utility of selection tests" by Kevin R. Murphy (Psychological Bulletin, 1986[Jan], Vol 99[1], 133-138). There was an error in the last line of Table 1, which appears on page 137. The corrected version is presented in the erratum. (The following abstract of the original article appeared in record 1986-15964-001.) Develops formulas for calculating the average ability of job candidates actually selected when the proportion of initial offers accepted is less than 100%. Three cases are considered: (a) those in which offers are declined at random, (b) those in which offers are declined by the highest-scoring applicants, and (c) those in which test scores are related to the probability of accepting an offer. It is shown that under realistic circumstances, utility formulas currently used could overestimate utility gains by 30–80% and that, under extreme circumstances, overestimation could be much worse. Such overestimation occurs even when tests are used only for screening rather than for rank-ordering candidates. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)