Validity of observer ratings of the big five personality factors.

Examined the validity of observer (supervisor, coworker, and customer) ratings and self-ratings of personality measures. Results based on a sample of 105 sales representatives supported the 2 hypotheses tested. First, supervisor, coworker, and customer ratings of the 2 job-relevant personality dimensions—conscientiousness and extraversion—were valid predictors of performance ratings, and the magnitude of the validities was at least as large as for self-ratings. Second, supervisor, coworker, and customer ratings accounted for significant variance in the criterion measure beyond self-ratings alone for the relevant dimensions. Overall, the results suggest that validities of personality measures based on self-assessments alone may underestimate the true validity of personality constructs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sex differences in validity for academic and employment criteria, and different types of predictors.

Sex differences in validity coefficients were examined for a total of 6,219 pairs of validities. Differences were also examined for subsets of predictors and criteria and a random sample of pairs of validities from independent data sets. Data were taken from studies in Educational and Psychological Measurement, the Journal of Applied Psychology, and Personnel Psychology from 1955 to the present and from the manuals of the Differential Aptitude Test Battery and the College Qualification Tests. Female validities were slightly but significantly higher than male validities for academic criteria and for all predictor subsets, with the exception of high school background and personality predictors. Female validities were also greater than male validities (.037 in correlation units) in the random sample of independent validity pairs. Male validities were slightly higher for high school background and personality predictors and for employment criteria. The generalizability of the finding concerning employment criteria and the latter 2 predictor subsets is questionable because of the small number of pairs of validities available and the nonindependence of the pairs that were available. (21 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of observer ratings of the five-factor model of personality traits: A meta-analysis.

Conclusions reached in previous research about the magnitude and nature of personality–performance linkages have been based almost exclusively on self-report measures of personality. The purpose of this study is to address this void in the literature by conducting a meta-analysis of the relationship between observer ratings of the five-factor model (FFM) personality traits and overall job performance. Our results show that the operational validities of FFM traits based on observer ratings are higher than those based on self-report ratings. In addition, the results show that when based on observer ratings, all FFM traits are significant predictors of overall performance. Further, observer ratings of FFM traits show meaningful incremental validity over self-reports of corresponding FFM traits in predicting overall performance, but the reverse is not true. We conclude that the validity of FFM traits in predicting overall performance is higher than previously believed, and our results underscore the importance of disentangling the validity of personality traits from the method of measurement of the traits. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Effects of impression management and self-deception on the predictive validity of personality constructs.

This study tests whether 2 types of response distortion (self-deception and impression management) affect the predictive validity of 2 of the "Big 5" personality dimensions, conscientiousness and emotional stability, in 2 applicant samples of long-haul semitruck drivers (n?=?147 and n?=?139). As hypothesized, conscientiousness (p?=?–.26 and –.26) and emotional stability (p?=?–.23 and –.21 ) were valid predictors of voluntary turnover in the 2 samples. Also as hypothesized, conscientiousness was a valid predictor of supervisory ratings of performance (p?=?.41 and .39 ) in the 2 samples. Although not hypothesized, emotional stability was also significantly related to supervisor ratings of performance (p?=?.23 and .27). Results from structural equations modeling indicated that applicants did distort their scores on both personality dimensions and the distortion occurred both through self-deception and impression management; however, neither type of distortion attenuated the predictive validities of either personality construct. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comprehensive meta-analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance.

Conducted a comprehensive meta-analysis based on 665 validity coefficients across 576,460 data points to investigate whether integrity test validities are generalizable and to estimate differences in validity due to potential moderating influences. Results indicate that integrity test validities are substantial for predicting job performance and counterproductive behaviors on the job, such as theft, disciplinary problems, and absenteeism. The estimated mean operational predictive validity of integrity test for predicting supervisory ratings of job performance is .41. Results from predictive validity studies conducted on applicants and using external criterion measures (i.e., excluding self-reports) indicate that integrity tests predict the broad criterion of organizationally disruptive behaviors better than they predict employee theft alone. Despite the influence of moderators, integrity test validities are positive across situations and settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Big Five Personality Traits and Individual Job Performance Growth Trajectories in Maintenance and Transitional Job Stages.

This study extends the literature on personality and job performance through the use of random coefficient modeling to test the validity of the Big Five personality traits in predicting overall sales performance and sales performance trajectories--or systematic patterns of performance growth--in 2 samples of pharmaceutical sales representatives at maintenance and transitional job stages (K. R. Murphy, 1989). In the maintenance sample, conscientiousness and extraversion were positively associated with between-person differences in total sales, whereas only conscientiousness predicted performance growth. In the transitional sample, agreeableness and openness to experience predicted overall performance differences and performance trends. All effects remained significant with job tenure statistically controlled. Possible explanations for these findings are offered, and theoretical and practical implications of findings are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Role of social desirability in personality testing for personnel selection: The red herring.

Response bias continues to be the most frequently cited criticism of personality testing for personnel selection. The authors meta-analyzed the social desirability literature, examining whether social desirability functions as a predictor for a variety of criteria, as a suppressor, or as a mediator. Social desirability scales were found not to predict school success, task performance, counterproductive behaviors, and job performance. Correlations with the Big Five personality dimensions, cognitive ability, and years of education are presented along with empirical evidence that (a) social desirability is not as pervasive a problem as has been anticipated by industrial-organizational psychologists, (b) social desirability is in fact related to real individual differences in emotional stability and conscientiousness, and (c) social desirability does not function as a predictor, as a practically useful suppressor, or as a mediator variable for the criterion of job performance. Removing the effects of social desirability from the Big Five dimensions of personality leaves the criterion-related validity of personality constructs for predicting job performance intact. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Are SSATS and GPA enough? A theory-based approach to predicting academic success in secondary school.

Two studies were carried out to predict academic success in the highly competitive environment of a private preparatory school, Choate Rosemary Hall. The 1st study focused on the question of whether there are indicators beyond middle school grade-point average (GPA) and standardized test scores that might enhance the validity of measures for predicting success of students attending Choate. The results indicated the importance of taking into account aspects of self-regulated learning (SRL), such as academic self-efficacy, academic motivation, academic locus of control, and measures of the WICS (Wisdom, Intelligence, Creativity Synthesized) theoretical framework. Both sets of SRL and WICS indicators demonstrated incremental validity in predicting success at Choate. The 2nd study preliminarily evaluated the value of including indicators of aspects of the SRL and the WICS theoretical framework into the Choate admission process. The results of this study examined the utility of using quantified indicators other than middle-school GPA and standardized test scores for making admission decisions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Early prospective studies of the high school dropout.

A limited number of empirical studies conducted between 1950 and 1970 used longitudinal designs to examine the relations between psychological characteristics of students and subsequent high school dropout. Five prospective studies demonstrated that modest but significant relations existed between completing high school and measures of personality gathered in the seventh, eighth or ninth grades. Stronger relations were found in three early studies that examined the link between sociometric measures gathered in late elementary school and high school graduation 6 or 7 years later. Results clarified some methodological requirements for useful research on school completion and suggested that social and emotional variables explain some of the variance in school dropout that is independent of academic performance. Implications are drawn for effective school dropout research, policies, and practices at the local, state, and national levels. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A closer look at the frame-of-reference effect in personality scale scores and validity.

This article contributes to the understanding of why the use of a frame-of-reference leads to increased criterion-related validity of personality inventories. Two competing explanations are described and tested. A between-subjects (N = 337) and a within-subject (N = 105) study are conducted to test the hypothesized effects of use of a frame of reference on reliability and validity. Regarding the effects on reliability, use of a frame of reference reduces within-person inconsistency (instead of between-person variability) in responding to generic items. Use of a frame of reference further leads to higher validity as a result of the reduction of between-person variability and within-person inconsistency. Yet, reducing these inconsistencies is not enough. It is also important to use a frame of reference that is conceptually relevant to the criterion. Besides implications for contextualized personality inventories, these results provide an explanation for the moderate validities of generic personality inventories. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Perfectionism, stress, and social (dis)connection: A short-term study of hopelessness, depression, and academic adjustment among honors students.

This study tested models of perfectionism predicting psychological distress and academic adjustment and moderators and mediators of those associations in 2 successive cohorts of high-achieving university honors students (N = 499). Participants completed measures early and late in the semester. Adaptive (high standards) and maladaptive (self-critical perceptions of inadequacy in meeting performance expectations) dimensions of perfectionism were found to be significantly associated, in generally expected directions, with concurrent and prospective perceived stress, social connectedness, depression, hopelessness, and perceived academic adjustment. However, some perfectionism effects were reduced when earlier psychological distress and adjustment were controlled in analyses predicting later distress and adjustment. Several effects were moderated and at least partially mediated by perceived stress and social connection. The results suggest several counseling implications for high-achieving, perfectionistic students. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Predicting success in college: A longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation.

The authors examined the role of achievement goals, ability, and high school performance in predicting academic success over students' college careers. First, the authors examined which variables predicted students' interest and performance in an introductory psychology course taken their first semester in college. Then, the authors followed students until they graduated to examine continued interest in psychology and performance in subsequent classes. Achievement goals, ability measures, and prior high school performance each contributed unique variance in predicting initial and long-term outcomes, but these predictors were linked to different educational outcomes. Mastery goals predicted continued interest, whereas performance-approach goals predicted performance. Ability measures and prior high school performance predicted academic performance but not interest. The findings support a multiple goals perspective. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The use of person-organization fit in employment decision making: An assessment of its criterion-related validity.

Because measures of person-organization (P-O) fit are accountable to the same psychometric and legal standards used for other employment tests when they are used for personnel decision making, the authors assessed the criterion-related validity of P-O fit as a predictor of job performance and turnover. Meta-analyses resulted in estimated true criterion-related validities of .15 (k = 36, N = 5,377) for P-O fit as a predictor of job performance and .24 (k = 8, N = 2,476) as a predictor of turnover, compared with a stronger effect of .31 (k = 109, N = 108,328) for the more commonly studied relation between P-O fit and work attitudes. In contrast to the relations between P-O fit and work attitudes, the lower 95% credibility values for the job performance and turnover relations included zero. In addition, P-O fit's relations with job performance and turnover were partially mediated by work attitudes. Potential concerns pertaining to the use of P-O fit in employment decision making are discussed in light of these results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Residual validity is not meaningful.

Comments on the article by R. J. Sternberg and W. M. Williams (see record 1997-04591-002) regarding the empirical validity of the Graduate Record Examination (GRE) in predicting graduate student performance in psychology. Sternberg and Williams claimed to have studied whether the GRE predicts meaningful success in graduate school, but they did not study GRE predictive validity for students applying to graduate school—they used students already selected for graduate school. Thus, it is argued, what Sternberg and Williams actually studied might be termed the residual validity of GRE scores—the validity remaining after some unknown amount of validity is exhausted in the admissions process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating the predictive validity of MCAT scores across diverse applicant groups

PURPOSE: To examine the predictive validity of MCAT scores, alone and in combination with other preadmission data, for medical students grouped by race/ethnicity and sex. METHOD: This study included two samples: 1,109 students who entered in 1992 any of the 14 medical schools participating in the MCAT Predictive Validity Study; and all 11,279 students who entered medical school in 1992 and took the USMLE Step 1 in June 1994. Criterion measures included each student's cumulative GPA in the first two years of medical school and his or her pass/fail status on Step 1. Differential predictive validity was examined by comparing prediction errors across racial/ethnic and sex groups. For cumulative GPA; residuals were compared, and for Step 1, classification errors were studied. RESULTS: The patterns of prediction errors observed across the groups indicated that, on average, (1) no difference between the sexes in prediction errors was evident; (2) performances of the three racial/ethnic minority groups tended to be overpredicted, with significant findings for Asians and Hispanics; and (3) Caucasians' performance tended to be underpredicted, although the magnitude of this underprediction was quite small. When USMLE Step 1 scores were the criterion for success in medical school, the majority of errors were overprediction errors. CONCLUSION: The authors caution that although MCAT scores, alone and in combination with undergraduate GPA, are good predictors of medical school performance, they are not perfect. The authors encourage future research exploring additional predictor variables, such as diligence, motivation, communication skills, study habits, and other relevant characteristics. Similarly, they indicate that high grades and Step 1 scores are not the only indicators of success in the medical profession and call for studies examining other important qualities, such as integrity, interpersonal skills, capacity for caring, willingness to commit to lifelong learning, and desire to serve in underserved areas.     

Testing the constancy of validity with computer-generated sampling distributions of the multiplicative model variance estimate: Results for petroleum industry validation research.

A comparison of computer-generated sampling distributions showed that increases in total N resulting from greater sample size/validity study reduced the sampling variation of multiplicative model estimates of true validity variance more than did increases in total N resulting from a larger number of studies. Validity coefficients, range restriction, and criterion reliability data for 5 tests predicting job performance and training performance were obtained from studies conducted at petroleum industry plants. Those tests included the Richardson-Bellows-Henry Ability, Arithmetic Reasoning, and Chemical Comprehension Tests; the Bennett Mechanical Comprehension Tests; and the Mathematics scale of the California Achievement Tests. Studies for 3 job categories (operation, maintenance, and laboratory) were combined for analysis to maximize sensitivity to job and plant differences. Sampling distributions of true validity variance estimates were generated for each test–criterion type combination. A comparison of actual variance estimates with critical values indicated that 3 of the 5 tests did not vary significantly in true validities for job performance or for training criteria. Results for 2 arithmetic reasoning tests support the existence of variation in true validities for job performance and for training criteria. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Setting, elaborating, and reflecting on personal goals improves academic performance.

Of students who enroll in 4-year universities, 25% never finish. Precipitating causes of early departure include poor academic progress and lack of clear goals and motivation. In the present study, we investigated whether an intensive, online, written, goal-setting program for struggling students would have positive effects on academic achievement. Students (N = 85) experiencing academic difficulty were recruited to participate in a randomized, controlled intervention. Participants were randomly assigned to 1 of 2 intervention groups: Half completed the goal-setting program, and half completed a control task with intervention-quality face validity. After a 4-month period, students who completed the goal-setting intervention displayed significant improvements in academic performance compared with the control group. The goal-setting program thus appears to be a quick, effective, and inexpensive intervention for struggling undergraduate students. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using theory to evaluate personality and job-performance relations: A socioanalytic perspective.

The authors used socioanalytic theory to understand individual differences in people's performance at work. Specifically, if predictors and criteria are aligned by using theory, then the meta-analytic validity of personality measures exceeds that of atheoretical approaches. As performance assessment moved from general to specific job criteria, all Big Five personality dimensions more precisely predicted relevant criterion variables, with estimated true validities of .43 (Emotional Stability), .35 (Extraversion-Ambition), .34 (Agreeableness), .36 (Conscientiousness), and .34 (Intellect-Openness to Experience). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of curtailment on an admissions model for a graduate management program.

Developed 2 admissions models to predict future academic performance of graduate management students. The 1st model was based on 40 students who were admitted and enrolled in the program and was uncorrected for curtailment. The 2nd model was developed from the total applicant population of 222 students after curtailment correction. The corrected model demonstrated higher predictive validity than the uncorrected model for 2 future classes of students. Furthermore, different predictors entered each model, affecting the beta weights, validities of predictors, and the total amount of variance explained by the models. A factor analysis and an analysis of admissions decisions offer additional support for the curtailment-corrected model in selecting students with high academic potential. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Measures of early academic skills: Reliability and validity with a first grade sample.

This study investigated the reliability- (interscorer and test-retest) and criterion-related validity (concurrent and predictive) of 11 curriculum-based measures of early academic skills with a sample of first-grade students. Some measures required production-type responses. Other measures required selection-type responses. The measures involved letters (sounds, names, copying), numbers (names, counting in sequence), colors, and shapes. Subjects were 30 first-grade students from an urban parochial school. Test-Retest reliability was assessed with alternate forms of the measures over a two-week interval. Concurrent validity was assessed with a norm-referenced achievement test (the Woodcock-Johnson Revised). Predictive validity was assessed by measuring oral reading fluency in passages and on word-lists four months after the initial assessment. Results suggested that some measures displayed better reliability and validity than others. The practical utility of these measures in light of their technical characteristics and the need for further construct validity data are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)