A simple solution.

Comments on the article by P. R. Sackett, N. Schmitt, J. E. Ellingson, and M. B. Kabin (see record 2001-00625-002) which posed the dilemma of how to use tests to select for performance without excluding minority group members. The current author argues that society should challenge the premise that elite American universities should select students on the basis of promise of academic performance. It is maintained that psychologists should develop valid tests that accurately predict the qualities of creativity, good citizenship, participation in extracurricular activities, athletics, political correctness, and social activism. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

No easy solution to subgroup differences.

Replies to comments by A. G. Bedeian (see record 2002-12457-016), G. Bocknek (see record 2002-12457-017), G. E. Zuriff (see record 2002-12457-018), and J. E. Helms (see record 2002-12457-019) on the article by P. R. Sackett, N. Schmitt, J. E. Ellingson, and M. B. Kabin (see record 2001-00625-002) which posed the dilemma of how to use tests to select for performance without excluding minority group members. The authors argue that it is difficult to envision a university that would ignore cognitive ability when making admissions decisions, as Zuriff advocated. In response to Bocknek, the authors question the validity of individual assessments to index traits such as motivation, perseverance, and problem solving. In response to Bedeian, the authors argue that society and the legal system (as it exists now) force the consideration of ascribed characteristics, and few, if any, good measures of racial identity (or intellectual diversity) exist. Finally, the authors argue that Helm's position that individuals' racial identity must be destroyed in the interest of raising their status in high-stakes testing situation is misguided. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

High stakes are an opportunity.

Comments on the article by P. R. Sackett et al (see record 2001-00625-002) which examines the use of cognitive testing in employment, credentialing, and higher education post-affirmative action. The current author states that Sackett et al made 2 badly flawed assumptions in their discussion. These assumptions are addressed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Application patterns when applicants know the odds: Implications for selection research and practice.

Unlike previous research that found small differences between population standard deviations and applicant pool standard deviations (P. R. Sackett & D. J. Ostgaard, 1994; D. S. Ones & C. Viswesvaran, 2003), this study revealed a 23% disparity between Law School Admission Test (LSAT) scores of all LSAT test takers and those of LSAT test takers who applied to law school. This study also illustrated robust applicant self-selection behavior across different law school ranks. These findings are important, because predictor scores of applicants who know their scores in advance and perceive small selection ratios necessitate substantially smaller range restriction corrections than those that would be required by population standard deviations. Furthermore, these findings more generally reveal that applicants who know their scores in advance behave quite differently from applicants who do not. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity and incremental validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations for predicting job performance in advanced-level high-stakes selection.

In high-stakes selection among candidates with considerable domain-specific knowledge and experience, investigations of whether high-fidelity simulations (assessment centers; ACs) have incremental validity over low-fidelity simulations (situational judgment tests; SJTs) are lacking. Therefore, this article integrates research on the validity of knowledge tests, low-fidelity simulations, and high-fidelity simulations in advanced-level high-stakes settings. A model and hypotheses of how these 3 predictors work in combination to predict job performance were developed. In a sample of 196 applicants, all 3 predictors were significantly related to job performance. Both the SJT and the AC had incremental validity over the knowledge test. Moreover, the AC had incremental validity over the SJT. Model tests showed that the SJT fully mediated the effects of declarative knowledge on job performance, whereas the AC partially mediated the effects of the SJT. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Reflections on the EPPP: A commentary on Sharpless and Barber.

In their article “The EPPP in the Era of Evidence-Based Practice,” B. A. Sharpless and J. P. Barber (see record 2009-11890-004) shine a critical spotlight on the standardized exam for professional psychology licensure. Yet their arguments are based on some inaccurate assumptions about the purpose and the validity of the Examination for Professional Practice in Psychology (EPPP). In this critical response paper, we focus on several areas of disagreement with Sharpless and Barber, including the purpose of the EPPP, reflections on its validity and failure rates, and the need for generic versus specialized entry-level professional licensure. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The Examination for Professional Practice in Psychology (EPPP) in the era of evidence-based practice.

Professional psychology has increasingly moved toward evidence-based practice. However, instruments used to assess psychologists seeking licensure, such as the Examination for Professional Practice in Psychology (EPPP), have received relatively little empirical scrutiny. Therefore, the authors evaluated the available evidence in support of the EPPP’s validity and current use as a core component of professional licensure. Although the EPPP has in many ways been extensively evaluated, there is a paucity of criterion, predictive, and incremental validity evidence available. Further, several aspects of the content validation studies were examined, and the authors question whether the EPPP, as currently constructed, can meet its stated goals. Given that the EPPP is a high-stakes examination and given the authors’ best estimate (based on a sample of 16 states) that 35% of applicants fail the examination, it is recommended that the EPPP be more extensively evaluated. An outline of major decision points in this proposed evaluation process is provided, several suggestions for further research are proposed, and the field is encouraged to discuss these issues further. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Racial/ethnic differences in the criterion-related validity of cognitive ability tests: A qualitative and quantitative review.

The correlation between cognitive ability test scores and performance was separately meta-analyzed for Asian, Black, Hispanic, and White racial/ethnic subgroups. Compared to the average White observed correlation ( = .33, N = 903,779), average correlations were lower for Black samples ( = .24, N = 112,194) and Hispanic samples ( = .30, N = 51,205) and approximately equal for Asian samples ( = .33, N = 80,705). Despite some moderating effects (e.g., type of performance criterion, decade of data collection, job complexity), validity favored White over Black and Hispanic test takers in almost all conditions that included a sizable number of studies. Black–White validity comparisons were possible both across and within the 3 broad domains that use cognitive ability tests for high-stakes selection and placement: civilian employment, educational admissions, and the military. The trend of lower Black validity was repeated in each domain; however, average Black–White validity differences were largest in military studies and smallest in educational and employment studies. Further investigation of the reasons for these validity differences is warranted. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Understanding the purpose, strengths, and limitations of the EPPP: A response to Sharpless and Barber.

Sharpless and Barber (see record 2009-11890-004) have written a thoughtful article questioning the validity, effectiveness, and transparency of the Examination for Professional Practice in Psychology (EPPP). This response has 3 goals: to clarify the purpose, structure, and validity of the EPPP; to provide some previously unavailable information that addresses concerns Sharpless and Barber raise about passing rates and assessing competence to practice; and to endorse some of the recommendations the authors make for future investigation and evolution of the psychology licensure examination process. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A manifest structure analysis of the Otis S-A Test of mental ability, higher examination: Form B.

This new scale theoretic approach to item analysis is shown to be useful in shortening a standardized test. A short test of 20 Otis items was developed. Testing time was 8 minutes. "Cross-validation resulted in a correlation of .82 between the scores obtained on the long and short forms." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Claude M. Steele: Award for Distinguished Senior Career Contributions to the Public Interest.

Provides the biography of Claude M. Steele and announces that he has won the Award for Distinguished Senior Career Contributions to the Public Interest for his influential and groundbreaking work on stereotype threat that has been critical in defining and documenting psychologists' understanding of how societal messages about identity and competency affect the performance and life decisions of racial/ethnic minority-group members and of girls and women. A selected bibliography is also provided. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Meta-Analytic Study of General Mental Ability Validity for Different Occupations in the European Community.

A comprehensive meta-analysis of the validity of general mental ability (GMA) measures across 12 occupational categories in the European Community (EC) is presented. GMA measures showed that there is validity generalization and large operational validities for job performance and training success in 11 occupational groups. Results also showed that job complexity moderated the magnitude of the operational validity of GMA tests across three levels of job complexity: low, medium, and high. In general, results were similar to those found in the United States, although the European findings showed a slightly larger magnitude of operational validity in some cases. Theoretical and practical implications of these findings for personnel selection are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A general formula for the quantitative treatment of human motivation.

"The simplest version of a comprehensive formula for the quantitative treatment of human motivation and its development was suggested." In words: the sum of intensity increments of motives that can be distinguished within a given person at a given time of development is equal to a constant. "It can be handled in practice by appropriate sampling and approximations. The major aspects of the formula were subjected to empirical tests. The formula proved feasible in all of the aspects investigated." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A biographical inventory for students: II. Validation of the instrument.

Scores on the BIS (see 31: 3079) were correlated, for samples of college freshmen, with 5 scholastic ability and achievement measures, 2 personality inventories, 2 values inventories, and vocational choice (major field of study). The magnitude and significance of the correlations between these 10 criteria and the 10 subscale scores of the BIS are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A note on the inadequacy of percentage grading scales for almost all ability distributions.

This study investigated distributions of letter grades (A,B,C,D,F) assigned to test scores according to the percentages adopted by Canadian universities, assuming realistic score distributions of ten different shapes with various means and standard deviations. The grade distributions corresponding to 83 out of 90 score distributions were highly anomalous, and the remaining 7 were far from ideal. Therefore, in the majority of practical testing situations, the percentage grading method is inadequate because of purely statistical properties of a scale based on fixed percentages. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Age-Based Structural Dynamics Between Perceptual Speed and Knowledge in the Berlin Aging Study: Direct Evidence for Ability Dedifferentiation in Old Age.

According to 2-component theories of intelligence, negative cross-sectional age gradients in mechanic (broad Gf) and pragmatic (broad Gc) cognitive components reflect the increasing constraining of the former in the expression and integrity of the latter component. The authors examined this widely held but untested assumption by applying a recently proposed dynamic structural modeling technique, the bivariate dual change score model, to longitudinal data from the Berlin Aging Study (N=516, age range=70-103 years). Mechanics and pragmatics were indexed by perceptual speed and knowledge, respectively. As hypothesized, results indicated that changes in knowledge are dominated by perceptual speed and offered strong support for the notion of "mechanization" of pragmatic abilities in old and very old age. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Further evidence for the validity of assessment center dimensions: A meta-analysis of the incremental criterion-related validity of dimension ratings.

This study investigates the incremental variance in job performance explained by assessment center (AC) dimensions over and above personality and cognitive ability. The authors extend previous research by using meta-analysis to examine the relationships between AC dimensions, personality, cognitive ability, and job performance. The results indicate that the 7 summary AC dimensions postulated by W. Arthur, Jr., E. A. Day, T. L. McNelly, & P. S. Edens (2003) are distinguishable from popular individual difference constructs and explain a sizeable proportion of variance in job performance beyond cognitive ability and personality. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A comparison of administration procedures for the Rey-Osterrieth Complex Figure: Flowcharts versus pen switching.

The Rey-Osterrieth Complex Figure (ROCF) is commonly used to assess visuospatial skills, visuoconstruction, visual memory, and executive functioning. Two different methods are traditionally used to record the order in which the figure is drawn: the flowchart method and the pen-switching method. Although it has been suggested that pen switching may interfere with performance, to date no research has been conducted to assess whether ROCF performance significantly differs due to administration method. As part of routine neuropsychological evaluation, 100 inpatients and outpatients were randomly assigned to either method. Using the Boston Qualitative Scoring System and the traditional 36-point scoring method, the authors unexpectedly found that the pen-switching group generally performed better than the flowchart group, and productions drawn with pen switching were also significantly faster to score. (PsycINFO Database Record (c) 2010 APA, all rights reserved)