Measurement of job and work involvement.

Developed separate measures of job and work constructs using 3 techniques: semantic differential, questionnaire, and graphic. Assessment measures included the Job Involvement and Work Involvement Questionnaires, Job Involvement and Work Involvement Semantic Differentials, and Job Involvement and Work Involvement Graphic scales. Data collected from a heterogeneous sample of 703 employees are analyzed to establish reliability, construct validity, and criterion-related validity of each measure. Results reveal that questionnaire and graphic measures pass the tests of reliability and validity. Semantic differential measures, however, have questionable validity for measuring work involvement and should, perhaps, be limited to only highly educated samples of respondents. Results also support the conceptual distinction between job and work involvement. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality scale validities increase throughout medical school.

Admissions and personnel decisions rely on stable predictor–criterion relationships. The authors studied the validity of Big Five personality factors and their facets for predicting academic performance in medical school across multiple years, investigating whether criterion-related validities change over time. In this longitudinal investigation, an entire European country’s 1997 cohort of medical students was studied throughout their medical school career (Year 1, N = 627; Year 7, N = 306). Over time, extraversion, openness, and conscientiousness factor and facet scale scores showed increases in operational validity for predicting grade point averages. Although there may not be any advantages to being open and extraverted for early academic performance, these traits gain importance for later academic performance when applied practice increasingly plays a part in the curriculum. Conscientiousness, perhaps more than any other personality trait, appears to be an increasing asset for medical students: Operational validities of conscientiousness increased from .18 to .45. In assessing the utility of personality measures, relying on early criteria might underestimate the predictive value of personality variables. Implications for personality measures to predict work performance are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Estimation of employment test validities by expert judgment.

Compared the relative accuracy of 2 methods of estimating employment test validity: expert judgment and small sample criterion-related validation studies. The study was based on US Navy data from samples of over 3,000 for each of 9 jobs, with validity results on 6 tests for each job. 20 experienced psychologists estimated the observed validity for each of the 54 test–job combinations. Both the random and systematic error in the expert judgments were evaluated. Psychologists typically underestimated the validity by a small amount (an average systematic error of .019). On the average, to equal the accuracy of a single judge, the sample size of a criterion-related validation study would have to be 92. To match the accuracy of an average across 4 judges, the sample size must be 326. The sample size must be 1,164 to match the accuracy of the pooled judgment of 30 judges. Results indicate that, given highly trained and experienced judges, expert judgment may provide more accurate estimates of validity for cognitive tests than do local criterion-related validation studies. (10 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Biographical data in employment selection: Can validities be made generalizable?

The hypothesis was examined that organizational specificity of biodata validity results from the methods typically used to select and key items. In this study, items were initially screened for job relevance, keying was based on large samples from multiple organizations, and items were retained only if they showed validity across organizations. Cross-validation was performed on approximately 11,000 first-line supervisors in 79 organizations. The resulting validities were meta-analyzed across organizations, age levels, sex, and levels of education, supervisory experience, and company tenure. In all cases, validities were generalizable. Validities were also stable across time and did not appear to stem from measurement of knowledge, skills, or abilities acquired through job experience. Finally, these results provide additional evidence against the hypothesis of situational specificity of validities, the first large-sample evidence in a noncognitive domain. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Investigator characteristics as moderators of personnel selection research: A meta-analysis.

Surrogate measures of original authors' reward context, primary motivation for doing the research, and knowledge, skills, and abilities (KSAs) were examined for their affect on criterion-related validities reported in the Journal of Applied Psychology and Personnel Psychology between 1964 and 1992. Number of years of experience (a surrogate KSA measure) displayed no moderating relationship. Type of organizational need (equal employment opportunity compliance, augmenting existing selection system, etc.) and investigator interests (e.g., theory testing) were related to criterion-related validities. Place of authors' employment (i.e., reward context) also displayed a moderating relationship (authors in private industry reported higher average validities in comparison with academics). Interaction effects on criterion-related validities were found between authors' experience and (1) place of employment and (2) primary motivation for conducting the research. Results are interpreted in view of possible differences in ability, motivation, and opportunity to do research across employment settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Biographical constructs for predicting employee suitability.

The use of background investigation data in personnel screening is reviewed. Background investigations are routinely conducted on persons seeking positions of trust in law enforcement, the nuclear power industry, and military and civilian occupations requiring government-issued security clearances. The application of background investigation information in personnel screening differs in many important ways from biodata applications developed by psychologists. In this article, these differences are reviewed, and the construct and criterion-related validity of a survey-based measure are examined. This measure taps content domains typically explored in background investigations. Seven background factors were identified. The background scales showed acceptable reliability, informative interscale relationships, and useful levels of criterion-related validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A study of item weights and scale lengths for the SVIB.

7 SVIB scales were developed and cross validated on 461 managers from 13 varied Minnesota companies. Questions studied were (a) Which item weighting method results in the highest scale validity? (b) Are shorter scales as valid as longer scales? (c) How much may scales be shortened? (d) Why may they be shortened? Controls for scale length, content, validity, and for item weighting method were introduced. Results indicated (a) there was no practical difference in validities between simple unit versus variably weighted scales, (b) shorter scales were as valid as longer scales, (c) Clark's "40 to 60 item optimum scale length" hypothesis was supported, (d) although not conclusive, shorter scales appeared superior partly because their average item validities were greater and thus they perhaps should not be used where developmental item pools are rich in valid items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of the validities of the Beck, Zung, and MMPI Depression scales.

Compared the validities of 3 widely used self-report depression measures: the Beck Depression Inventory (BDI), the MMPI Depression scale, and the Self-Rating Depression Scale (SRDS). Each inventory was administered to 101 inpatient psychiatric ward patients and to 99 chemical dependency ward patients. All of the Ss were male and less than 60 yrs of age. The 3 scales were correlated with clinicians' global ratings of depression, with scores on 5 DSM-III-based factor-analytic depression scales, and with an overall depression score based on the DSM-III criteria. In general, the SRDS produced better validity coefficients than the BDI, which in turn yielded higher correlations with these criteria than did the MMPI Depression Scale. (5 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity issues in clinical assessment.

The data investigators should provide to support the validity of inferences they make based on scores from a measure depend on (a) whether the measure is assumed to assess a hypothetical construct or behavior and (b) the purposes for which the measure is intended. The authors distinguish between the representational phase of validity assessment, which establishes that a measure produces scores that reflect the construct or behavior it purports to assess, and the elaborative validity phase, in which the meaning and utility of scores are examined. Key issues relevant to convergent, discriminant, and criterion-related validity are examined for measures of latent traits or constructs and then for measures of behaviors or response classes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of factor structures and criterion-related validity coefficients for two measures of personality based on the five factor model.

The factor structure and criterion-related validity for 2 types of personality measures that are based on 5 factor models were studied. Confirmatory factor analysis was used to compare a 5 versus 6 factor model in an applied setting. In addition, criterion-related validity coefficients were examined for the 2 inventories. Two Big Five measures were used—1 was an adjective-based bipolar inventory and the other was a questionnaire (NEO-Personality Inventory; PI)—to shed light on the relationships between inventory characteristics, factor structure, and criterion-related validity. The sample consisted of 423 flight attendant trainees. Results indicated that the 6 factor model provided a better fit for both measures compared with the 5 factor model. Scales from the NEO-PI were significantly correlated with measures of training success, whereas scales from the bipolar inventory were not. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance.

Conducted a comprehensive meta-analysis based on 665 validity coefficients across 576,460 data points to investigate whether integrity test validities are generalizable and to estimate differences in validity due to potential moderating influences. Results indicate that integrity test validities are substantial for predicting job performance and counterproductive behaviors on the job, such as theft, disciplinary problems, and absenteeism. The estimated mean operational predictive validity of integrity test for predicting supervisory ratings of job performance is .41. Results from predictive validity studies conducted on applicants and using external criterion measures (i.e., excluding self-reports) indicate that integrity tests predict the broad criterion of organizationally disruptive behaviors better than they predict employee theft alone. Despite the influence of moderators, integrity test validities are positive across situations and settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Mini-IPIP Scales: Tiny-yet-effective measures of the Big Five Factors of Personality.

The Mini-IPIP, a 20-item short form of the 50-item International Personality Item Pool-Five-Factor Model measure (Goldberg, 1999), was developed and validated across five studies. The Mini-IPIP scales, with four items per Big Five trait, had consistent and acceptable internal consistencies across five studies (= at or well above .60), similar coverage of facets as other broad Big Five measures (Study 2), and test-retest correlations that were quite similar to the parent measure across intervals of a few weeks (Study 4) and several months (Study 5). Moreover, the Mini-IPIP scales showed a comparable pattern of convergent, discriminant, and criterion-related validity (Studies 2-5) with other Big Five measures. Collectively, these results indicate that the Mini-IPIP is a psychometrically acceptable and practically useful short measure of the Big Five factors of personality. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Item selection counts: A comparison of empirical key and rational scale validities in theory-based and non-theory-based item pools.

Little explicit attention has been given to the impact of item pools on the validities and cross-validities of different background data scoring approaches. This study tests the idea that pools of items theoretically related to the performance of interest will outperform pools of items with no hypothesized relationship with the criterion. Validities and cross-validities of rational scales and empirical keys created from theory- and non-theory-based item pools were compared for 3 criteria. When size of the item pools was held constant, theory-based empirical keys (correlational and vertical percent) and rational scales showed larger validities and cross-validities than non-theory-based empirical keys (correlational and vertical percent) and showed minimal shrinkage in cross-validities. Even when item pool for the non-theory-based keys was expanded to include all items in the instrument, the theory-based keys showed comparable or slightly better validities and cross-validities for 2 of the 3 criteria, including college GPA, which was separated from the predictors by 4 years. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multidimensional assessment of parent-adolescent relations.

The Parent–Adolescent Relationship Questionnaire (PARQ) was constructed to be a multidimensional self-report inventory consisting of 16 scales assessing problem-solving communication skills, beliefs/attributions, and family structure. The PARQ was administered to a heterogeneous sample of 577 families in a test of its internal consistency, factor structure, and criterion-related validity. Internal consistency was generally above .80. Factor analyses yielded 3 factors corresponding to the hypothetically deduced constructs of skill deficits, beliefs, and family structure. Criterion-related validity tests contrasting families with acting-out, behavior-disordered adolescents, well-adjusted adolescents, and a broad normative group indicated that the Skill Deficit and Beliefs scales were sensitive indicators of level of distress but that the Family Structure scales did not reliably distinguish between the groups. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Impact of Elaboration on Socially Desirable Responding and the Validity of Biodata Measures.

The current study investigated the impact of requiring respondents to elaborate on their answers to a biodata measure on mean scores, the validity of the biodata item composites, subgroup mean differences, and correlations with social desirability. Results of this study indicate that elaborated responses result in scores that are much lower than nonelaborated responses to the same items by an independent sample. Despite the lower mean score on elaborated items, it does not appear that elaboration affects the size of the correlation between social desirability and responses to biodata items or that it affects criterion-related validity or subgroup mean differences in a practically significant way. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of observer ratings of the big five personality factors.

Examined the validity of observer (supervisor, coworker, and customer) ratings and self-ratings of personality measures. Results based on a sample of 105 sales representatives supported the 2 hypotheses tested. First, supervisor, coworker, and customer ratings of the 2 job-relevant personality dimensions—conscientiousness and extraversion—were valid predictors of performance ratings, and the magnitude of the validities was at least as large as for self-ratings. Second, supervisor, coworker, and customer ratings accounted for significant variance in the criterion measure beyond self-ratings alone for the relevant dimensions. Overall, the results suggest that validities of personality measures based on self-assessments alone may underestimate the true validity of personality constructs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Moderating effects of decision-making/information-processing job dimensions on test validities.

Analyzed selection test data from the General Aptitude Test Battery and job dimension data from the Position Analysis Questionnaire for 111 diverse jobs. Test validities were statistically corrected for restriction of range for each job and correlated with each of 3 information-processing/decision-making (IP/DM) job dimensions and 2 manually oriented job dimensions to determine possible moderating effects of these dimensions. Results show that the IP/DM dimensions generally moderated the validities of the general intelligence test, the verbal ability test, and the numerical test, with positive correlations being demonstrated, and the validities of the finger and manual dexterity tests, with negative correlations demonstrated. The manually oriented job dimensions did not, however, reveal any significant moderating effects. Results are discussed in terms of their implications for the validity generalization model extended by F. L. Schmidt and J. E. Hunter (see record 1978-11448-001). (21 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Synthetic validity: A conceptual and comparative review.

Describes synthetic validity as the logical process of inferring test-battery validity from predetermined validities of the tests for basic work components. Two programmatic approaches to synthetic validity are discussed: the J-coefficient and the job component model. The J-coefficient is a mathematical index of the test and job-performance relationships that has been shown to yield results comparable to intertest and criterion correlation methods of assembling test batteries. The job component model associates mean test-scores with job components and is related to criterion-related validation while the J-coefficient approach is more related to content validation. Other approaches have used ratings as criteria for component validities, but they must be free of the usual rater biases to be useful. It is suggested that the most important aspect of synthetic validity is in making selection decisions. (55 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Statistical power in criterion-related validation studies.

Examined systematically the sample sizes necessary to provide adequate power in validation studies under various conditions of range restriction and criterion unreliability. For purposes of brevity, the examination was restricted to the validity parameter values (i.e., true validities) of .35 and .50. Results demonstrate that sample sizes required to produce adequate power in empirical validation studies are substantially larger than has typically been assumed. This finding leads to the conclusion that, from the viewpoint of sample-size requirements, criterion-related validity studies are "technically feasible" much less frequently than is commonly assumed. It is also shown that the "situational specificity" of employment test validities may be in large part a consequence of excessive faith in small-sample analyses, that is, belief in "the law of small numbers." (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)