Responses to issues raised about validity, bias, and fairness in high-stakes testing.

We are pleased that our article (see record 2008-05553-001) prompted this series of four commentaries and that we have this opportunity to respond. We address each in turn. Duckworth (see record 2009-06923-012) and Kaufman and Agars (see record 2009-06923-013) discussed, respectively, two broad issues concerning the validity of selection systems, namely, the expansion of the predictor domain to include noncognitive predictors of performance and the expansion of the criterion domain to include additional criteria (e.g., creativity). We agree with these arguments, noting that they expand on points made in our original article. Wicherts and Millsap (see record 2009-06923-014) rightly noted the distinction between measurement bias and predictive bias and the fact that a finding of no predictive bias does not rule out the possibility that measurement bias still exists. They took issue with a statement we cited from Cullen, Hardison, and Sackett (2004) that if motivational mechanisms, such as stereotype threat, result in minority group members obtaining lower observed scores than true scores (i.e., a form of measurement bias), then the performance of minority group members should be under predicted. Our characterization of Cullen et al.’s (2004) statement was too cryptic; what was intended was a statement to the effect that if the regression lines for majority and minority groups are identical at the level of true predictor scores, then a biasing factor resulting in lower observed scores than true scores for minority group members would shift the minority group regression line to result in under prediction for that group. We do agree with Helms’s (see record 2009-06923-015) call for studying the reasons why racial- group differences are found and encourage this line of research; however, we view the study of racial-group differences and the study of determinants of those differences as complementary. We thank the authors for contributing these commentaries and for stimulating this discussion. Duckworth (2009) and Kaufman and Agars (2009) discussed important issues regarding expanding the predictor and criterion domains. Wicherts and Millsap (2009) correctly noted distinctions between predictive and measurement bias and used stereotype threat as a mechanism to discuss these issues. Helms (2009) raised several issues regarding the validity and fairness of standardized tests. In all cases, we welcomed the opportunity to discuss these topics and provide more detail on issues relating to high-stakes standardized testing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of response instructions on situational judgment test performance and validity in a high-stakes context.

This study fills a key gap in research on response instructions in situational judgment tests (SJTs). The authors examined whether the assumptions behind the differential effects of knowledge and behavioral tendency SJT response instructions hold in a large-scale high-stakes selection context (i.e., admission to medical college). Candidates (N = 2,184) were randomly assigned to a knowledge or behavioral tendency response instruction SJT, while SJT content was kept constant. Contrary to prior research in low-stakes settings, no meaningfully important differences were found between mean scores for the response instruction sets. Consistent with prior research, the SJT with knowledge instructions correlated more highly with cognitive ability than did the SJT with behavioral tendency instructions. Finally, no difference was found between the criterion-related validity of the SJTs under the two response instruction sets. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Gravitation to jobs commensurate with ability: Longitudinal and cross-sectional tests.

Two large data sets were used to test the premise that individuals will, over time, gravitate into jobs commensurate with their abilities. First, in a longitudinal data set with the individual as the unit of analysis, cognitive ability predicted movement in a job hierarchy over a 5-year period. Second, in a cross-sectional data set with the job as the unit of analysis, groups made up of less experienced incumbents were found to be slightly less homogeneous with respect to cognitive ability than groups made up of more experienced incumbents. Thus, support for the gravitational hypothesis was found. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The point of PDM [product data management]

Manufacturing industry's success in reforming itself has exposed an underlying factor limiting further significant gains in competitive performance; the effective management of product data. The growth of computer-based systems has led to the existence of a large amount of product definition data, stored in various locations and formats. To achieve the proper management of these data, product data management (PDM) has evolved. The authors discuss the aims and benefits of PDM     

Extreme Adiabatic Expansion in Micro-gravity: Modeling for the Cold Atomic Laboratory

The upcoming Cold Atom Laboratory mission for the International Space Station will allow the investigation of ultracold gases in a microgravity environment. Cold atomic samples will be produced using evaporative cooling in a magnetic chip trap. We investigate here the possibility to release atoms from the trap via adiabatic expansion. We discuss both general considerations and a detailed model of the planned apparatus. We find that it should be possible to reduce the mean trap confinement frequency to about 0.2 Hz, which will correspond to a three-dimensional sample temperature of about 150 pK and a mean atom velocity of 0.1 mm/s.     

Xenon and krypton as radiographic inhalation contrast media with computerized tomography: preliminary note

Uptake of D-glucose anomers by isolated rat retina was studied. After 3 min incubation at 37 degrees C in the presence of alpha or beta anomer (750 mug/ml), a significantly greater uptake (1.32 mg/g wet tissue) of beta-anomer was observed compared with that of alpha-D-glucose (1.11 mg/g wet tissue). This result and other data suggest that the carrier for D-glucose transport in the retina prefers the beta-anomer stereospecifically.     

On the Value of Correcting Mischaracterizations of Stereotype Threat Research.

Replies to comments by C. M. Steele and J. Aronson (see record 2004-10043-009) on the mischaracterizations alleged by P. R. Sackett et al (see record 2004-10043-001) in the 1995 experiment by Steele and Aronson (see record 1996-12938-001) on the intellectual test performance of African Americans. (PsycINFO Database Record (c) 2010 APA, all rights reserved)