Testing the constancy of validity with computer-generated sampling distributions of the multiplicative model variance estimate: Results for petroleum industry validation research.

A comparison of computer-generated sampling distributions showed that increases in total N resulting from greater sample size/validity study reduced the sampling variation of multiplicative model estimates of true validity variance more than did increases in total N resulting from a larger number of studies. Validity coefficients, range restriction, and criterion reliability data for 5 tests predicting job performance and training performance were obtained from studies conducted at petroleum industry plants. Those tests included the Richardson-Bellows-Henry Ability, Arithmetic Reasoning, and Chemical Comprehension Tests; the Bennett Mechanical Comprehension Tests; and the Mathematics scale of the California Achievement Tests. Studies for 3 job categories (operation, maintenance, and laboratory) were combined for analysis to maximize sensitivity to job and plant differences. Sampling distributions of true validity variance estimates were generated for each test–criterion type combination. A comparison of actual variance estimates with critical values indicated that 3 of the 5 tests did not vary significantly in true validities for job performance or for training criteria. Results for 2 arithmetic reasoning tests support the existence of variation in true validities for job performance and for training criteria. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity generalization in the context of situational models.

A primary objective of validity generalization (VG) analysis is to decompose the between-situation variance in validities into (1) variance attributable to between-situation differences in statistical artifacts and (2) variance attributable to between-situation differences in (unidentified) situational moderators. This process is based on the assumption that the effects of statistical artifacts on validities are independent of the effects of situational moderators on validities. The present article seeks to question the independence assumption by theoretically integrating situational variables into the VG estimation process. It is shown that the independence assumption may be untenable because at least 1 artifact (criterion reliability) is a function of, rather than independent of, the situational variables that moderate validities. An alternative approach to VG analysis is recommended. This approach rests heavily on proactive research designs in which potential situational moderators are included in the generalizability analysis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Extension of the Schmidt and Hunter validity generalization procedure to the prediction of absenteeism behavior from knowledge of job satisfaction and organizational commitment.

Examined whether the weak relationship between job satisfaction and absenteeism could be explained by the statistical artifacts of I. L. Schmidt et al (see record 1978-11448-001) (criterion unreliability, predictor unreliability, range restriction, sampling error, computational and typographical errors, criterion contamination and deficiency, and slight differences in factor structures). Data were collected on job satisfaction and organizational commitment from 242 sales employees. Evidence of differences were found on variable means, standard deviations, reliabilities, and predictive validities. However, when 6 artifacts were controlled, little variability in validities remained. (32 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development and validation of minicourses in the telecommunication industry.

Beginning in the early 1970s, American Telephone and Telegraph (AT&T) and the associated Bell companies have used minicourses for selecting personnel in a variety of jobs involving new technologies. The minicourse approach uses material representative of training content to assess the candidate's ability to acquire knowledge relevant to full-scale training and the job. A major advantage of the minicourse approach is the ability to implement a selection procedure concurrent with the introduction of a new training program. Although minicourses are initially validated using a content-oriented strategy, the AT&T model has included criterion-related validation where possible. The experience of AT&T with seven minicourses is discussed, and validities for training criteria and job performance are reported. Minicourses are a specific example of the general class of predictors known as trainability tests. The results of a meta-analysis indicate that validities for trainability tests are comparable with validities found for ability tests and work sample tests by Hunter and Hunter (1984). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Differentiating specific job knowledge from implicit trait policies in procedural knowledge measured by a situational judgment test.

This study compared validities of situational judgment test (SJT) scoring keys that were presumed to be differentially saturated with specific knowledge about effective job behavior and general knowledge about effective trait expression. The keys were based on subject matter experts’ effectiveness judgments, undergraduates’ effectiveness judgments, and graduate students’ trait judgments. We used data reported earlier by Motowidlo, Dunnette, and Carter (1990) with managerial incumbents in telecommunication companies. All keys yielded valid relations with supervisory performance ratings. The key based on subject matter experts’ judgments, however, explained criterion variance beyond the variance explained by the other keys. These results suggest that specific knowledge about effective job behavior and general knowledge about effective trait expression (i.e., implicit trait policies) contribute independently to variance in job performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Positive manifold limits the relevance of content-matching strategies for validating selection test batteries.

Inferences about the relationships between scores on selection tests and measures of job performance are often made on the basis of an assessment of the match between the content of the test and the content of the job. However, there is little evidence that these test-to-job comparisons have any bearing on the criterion-related validity of selection tests. The authors show that conclusions reached in analyses of cognitive tests—that content matching is largely irrelevant to criterion-related validity—can be generalized to most sets of selection tests (e.g., psychomotor and performance tests, interview ratings, biodata scores, knowledge tests, work sample tests) that are positively correlated with one another and with the criterion. When the universe of potential predictors shows positive manifold, almost all possible sets of test batteries will yield similar outcomes and show similar validities, regardless of whether the content of these tests matches the content of the job. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Predicting absenteeism and turnover: A field comparison of Fishbein's model and traditional job attitude measures.

Studied the lab-to-field generalizability of M. Fishbein's (1967) attitude-behavior model and examined the relative efficacy of the Fishbein model and traditional job attitude measures as predictors of absenteeism and turnover. Predictor data were collected from 108 nursing home employees immediately preceding the 2-mo time period of interest. Criterion data were obtained at the end of the 2-mo period. Fishbein's model received some field support, particularly with respect to predicting turnover. Traditional job attitude measures were more effective predictors of absenteeism, while Fishbein's model was a more effective predictor of turnover. It is concluded that neither approach seems superior especially in light of the amounts of criterion variance accounted for. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Meta-Analytic Study of General Mental Ability Validity for Different Occupations in the European Community.

A comprehensive meta-analysis of the validity of general mental ability (GMA) measures across 12 occupational categories in the European Community (EC) is presented. GMA measures showed that there is validity generalization and large operational validities for job performance and training success in 11 occupational groups. Results also showed that job complexity moderated the magnitude of the operational validity of GMA tests across three levels of job complexity: low, medium, and high. In general, results were similar to those found in the United States, although the European findings showed a slightly larger magnitude of operational validity in some cases. Theoretical and practical implications of these findings for personnel selection are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analysis of assessment center validity.

Meta-analysis (Hunter, Schmidt, & Jackson, 1982) of 50 assessment center studies containing 107 validity coefficients revealed a corrected mean and variance of .37 and .017, respectively. Validities were sorted into five categories of criteria and four categories of assessment purpose. Higher validities were found in studies in which potential ratings were the criterion, and lower validities were found in promotion studies. Sufficient variance remained after correcting for artifacts to justify searching for moderators. Validities were higher when the percentage of female assessees was high, when several evaluation devices were used, when assessors were psychologists rather than managers, when peer evaluation was used, and when the study was methodologically sound. Age of assessees, whether feedback was given, days of assessor training, days of observation, percentages of minority assessees, and criterion contamination did not moderate assessment center validities. The findings suggest that assessment centers show both validity generalization and situational specificity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Demographic, clinical, and neurocognitive correlates of everyday functional impairment in severe mental illness.

Although cognitive deficits often accompany severe mental illness, their implications for everyday functioning remain poorly understood. In this study, an occupational therapist (OT) rated the everyday functioning of 105 adult psychiatric patients. Using demographic, clinical, and cognitive variables, the authors tested alternative models to account for the observed variability in OT ratings. Although age, education, and the presence of schizophrenia each contributed to a model that accounted for 27% of the variation in functional independence, adding terms for auditory divided attention and verbal learning increased the proportion of explained variance to 45% and decreased the beta weights for age and education—but not schizophrenia—to nonsignificant levels. These findings demonstrate the relevance of cognitive performance to everyday functioning in severe mental illness. They are discussed with respect to hypothesized determinants of psychiatric disability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing common sense.

Predictors of success in school, such as conventional psychometric intelligence (e.g., IQ) tests, are less predictive of success out of school. Even the most charitable estimates of the relation between intelligence test scores and real-world criteria such as job performance indicate that approximately three fourths of the variance in real-world performance is not accounted for by intelligence test performance. Researchers have begun to explore new constructs in search of measures to supplement existing cognitive ability tests as predictors of real-world performance. Among the most promising constructs is practical intelligence, or common sense. Performance on measures of practical intelligence predicts real-world criteria such as job performance but is relatively unrelated to performance on intelligence tests and other common selection measures. Consequently, its contribution to prediction is largely independent of the contributions of existing measures, including measures of cognitive ability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of curtailment on an admissions model for a graduate management program.

Developed 2 admissions models to predict future academic performance of graduate management students. The 1st model was based on 40 students who were admitted and enrolled in the program and was uncorrected for curtailment. The 2nd model was developed from the total applicant population of 222 students after curtailment correction. The corrected model demonstrated higher predictive validity than the uncorrected model for 2 future classes of students. Furthermore, different predictors entered each model, affecting the beta weights, validities of predictors, and the total amount of variance explained by the models. A factor analysis and an analysis of admissions decisions offer additional support for the curtailment-corrected model in selecting students with high academic potential. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance.

Conducted a comprehensive meta-analysis based on 665 validity coefficients across 576,460 data points to investigate whether integrity test validities are generalizable and to estimate differences in validity due to potential moderating influences. Results indicate that integrity test validities are substantial for predicting job performance and counterproductive behaviors on the job, such as theft, disciplinary problems, and absenteeism. The estimated mean operational predictive validity of integrity test for predicting supervisory ratings of job performance is .41. Results from predictive validity studies conducted on applicants and using external criterion measures (i.e., excluding self-reports) indicate that integrity tests predict the broad criterion of organizationally disruptive behaviors better than they predict employee theft alone. Despite the influence of moderators, integrity test validities are positive across situations and settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Moderating effects of decision-making/information-processing job dimensions on test validities.

Analyzed selection test data from the General Aptitude Test Battery and job dimension data from the Position Analysis Questionnaire for 111 diverse jobs. Test validities were statistically corrected for restriction of range for each job and correlated with each of 3 information-processing/decision-making (IP/DM) job dimensions and 2 manually oriented job dimensions to determine possible moderating effects of these dimensions. Results show that the IP/DM dimensions generally moderated the validities of the general intelligence test, the verbal ability test, and the numerical test, with positive correlations being demonstrated, and the validities of the finger and manual dexterity tests, with negative correlations demonstrated. The manually oriented job dimensions did not, however, reveal any significant moderating effects. Results are discussed in terms of their implications for the validity generalization model extended by F. L. Schmidt and J. E. Hunter (see record 1978-11448-001). (21 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Joint relation of experience and ability with job performance: Test of three hypotheses.

Data from four different jobs (N?=?1,474) were used to evaluate three hypotheses of the joint relation of job experience and general mental ability to job performance as measured by (a) work sample measures, (b) job knowledge measures, and (c) supervisory ratings of job performance. The divergence hypothesis predicts an increasing difference and the convergence hypothesis predicts a decreasing difference in the job performance of high- and low-mental-ability employees as employees gain increasing experience on the job. The noninteractive hypothesis, by contrast, predicts that the performance difference will be constant over time. For all three measures of job performance, results supported the noninteractive hypothesis. Also, consistent with the noninteractive hypothesis, correlational analyses showed essentially constant validities for general mental ability (measured earlier) out to 5 years of experience on the job. In addition to their theoretical implications, these findings have an important practical implication: They indicate that the concerns that employment test validities may decrease over time, complicating estimates of selection utility, are probably unwarranted. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Creativity tests and achievement in high school science.

This study investigated the relative validities of a battery of "creativity tests" and an IQ tests for predicting several indices of achievement in high school science. Criteria included grade-point average in science courses, percentile rank on the STEP Science Achievement Test, teacher rating of overall scientific potential, number of high school science courses taken, and a measure of involvement with science. Results indicated that the creativity tests did have considerable predictive validity against each criterion for each sex and that the criterion variance accounted for by the creativity tests is to a substantial degree independent of IQ. Contrary to findings of other investigators, teachers did not discriminate against highly creative pupils in their ratings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Gender differences in cognitive development.

The potential effect of gender on intellectual abilities remains controversial. The purpose of this research was to analyze gender differences in cognitive test performance among children from continuous age groups. For this purpose, the normative data from 7 domains of the newly developed neuropsychological test battery, the Evaluación Neuropsicológica Infantil [Child Neuropsychological Assessment] (Matute, Rosselli, Ardila, & Ostrosky-Solis, 2007), were analyzed. The sample included 788 monolingual children (350 boys, 438 girls) ages 5 to 16 years from Mexico and Colombia. Gender differences were observed in oral language (language expression and language comprehension), spatial abilities (recognition of pictures seen from different angles), and visual (Object Integration Test) and tactile perceptual tasks, with boys outperforming girls in most cases, except for the tactile tasks. Gender accounted for only a very small percentage of the variance (1%–3%). Gender × Age interactions were observed for the tactile tasks only. It was concluded that gender differences during cognitive development are minimal, appear in only a small number of tests, and account for only a low percentage of the score variance. (PsycINFO Database Record (c) 2011 APA, all rights reserved)