Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences.

This article analyzes latent variable models from a cognitive psychology perspective. We start by discussing work by Tuerlinckx and De Boeck (2005), who proved that a diffusion model for 2-choice response processes entails a 2-parameter logistic item response theory (IRT) model for individual differences in the response data. Following this line of reasoning, we discuss the appropriateness of IRT for measuring abilities and bipolar traits, such as pro versus contra attitudes. Surprisingly, if a diffusion model underlies the response processes, IRT models are appropriate for bipolar traits but not for ability tests. A reconsideration of the concept of ability that is appropriate for such situations leads to a new item response model for accuracy and speed based on the idea that ability has a natural zero point. The model implies fundamentally new ways to think about guessing, response speed, and person fit in IRT. We discuss the relation between this model and existing models as well as implications for psychology and psychometrics. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The dimensionality of phonological awareness: An application of item response theory.

A battery of 7 tasks composed of 105 items thought to measure phonological awareness skills was administered to 945 children in kindergarten through 2nd grade. Results from confirmatory factor analysis at the task level and modified parallel analysis at the item level indicated that performance on these tasks was well represented by a single latent dimension. A 2-parameter logistic item response (IRT) model was also fit to the performance on the 105 items. Information obtained from the IRT model demonstrated that the tasks varied in the information they provided about a child's phonological awareness skills. These results showed that phonological awareness, as measured by these tasks, appears to be well represented as a unidimensional construct, but the tasks best suited to measure phonological awareness vary across development. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A nonlinear mixed model framework for item response theory.

Mixed models take the dependency between observations based on the same cluster into account by introducing 1 or more random effects. Common item response theory (IRT) models introduce latent person variables to model the dependence between responses of the same participant. Assuming a distribution for the latent variables, these IRT models are formally equivalent with nonlinear mixed models. It is shown how a variety of IRT models can be formulated as particular instances of nonlinear mixed models. The unifying framework offers the advantage that relations between different IRT models become explicit and that it is rather straightforward to see how existing IRT models can be adapted and extended. The approach is illustrated with a self-report study on anger. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development of a new critical thinking test using item response theory.

The authors describe the initial development of the Wagner Assessment Test (WAT), an instrument designed to assess critical thinking, using the 5-faceted view popularized by the Watson-Glaser Critical Thinking Appraisal (WGCTA; G. B. Watson & E. M. Glaser, 1980). The WAT was designed to reduce the degree of successful guessing relative to the WGCTA by increasing the number of response alternatives (i.e., 80% of WGCTA items are 2-alternative, multiple-choice), a change that was hypothesized to result in more desirable test information and standard-error functions. Analyses using the 3-parameter logistic item response theory (IRT) model in a sample of undergraduates (N = 407) supported this prediction, even when the WAT item pool was shortened to match the length of the WGCTA. Convergent validity between full-pool IRT score estimates was r = .69. Implications for subsequent research on IRT-based measurement of critical thinking are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Measurement of alcohol-related consequences among high school and college students: Application of item response models to the Rutgers Alcohol Problem Index.

The Rutgers Alcohol Problem Index (RAPI; H. R. White & E. W. Labouvie, 1989) is a frequently used measure of alcohol-related consequences in adolescents and college students, but psychometric evaluations of the RAPI are limited and it has not been validated with college students. This study used item response theory (IRT) to examine the RAPI on students (N = 895; 65% female, 35% male) assessed in both high school and college. A series of 2-parameter IRT models were computed, examining differential item functioning across gender and time points. A reduced 18-item measure demonstrating strong clinical utility is proposed, with scores of 8 or greater implying greater need for treatment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Diagnosing item score patterns on a test using item response theory-based person-fit statistics.

Person-fit statistics have been proposed to investigate the fit of an item score pattern to an item response theory (IRT) model. The author investigated how these statistics can be used to detect different types of misfit. Intelligence test data were analyzed using person-fit statistics in the context of the G. Rasch (1960) model and R. J. Mokken's (1971, 1997) IRT models. The effect of the choice of an IRT model to detect misfitting item score patterns and the usefulness of person-fit statistics for diagnosis of misfit are discussed. Results showed that different types of person-fit statistics can be used to detect different kinds of person misfit. Parametric person-fit statistics had more power than nonparametric person-fit statistics. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using IRT to separate measurement bias from true group differences on homogeneous and heterogeneous scales: An illustration with the MMPI.

The authors present a didactic illustration of how item response theory (IRT) can be used to separate measurement bias from true group differences on homogeneous and heterogeneous scales. Several bias detection methods are illustrated with 12 unidimensional Minnesota Multiphasic Personality Inventory (MMPI) factor scales (Waller, 1999) and the 13 multidimensional MMPI validity and clinical scales. The article begins with a brief review of MMPI bias research and nontechnical reviews of the 2-parameter logistic model (2-PLM) and several IRT-based methods for bias detection. A goal of this article is to demonstrate that homogeneous and heterogeneous scales that are composed of biased items do not necessarily yield biased test scores. To that end, the authors perform differential item- and test-functioning analyses on the MMPI factor, validity, and clinical scales using data from 511 Blacks and 1,277 Whites from the California Youth Authority. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy.

In this article, the authors developed a common strategy for identifying differential item functioning (DIF) items that can be implemented in both the mean and covariance structures method (MACS) and item response theory (IRT). They proposed examining the loadings (discrimination) and the intercept (location) parameters simultaneously using the likelihood ratio test with a free-baseline model and Bonferroni corrected critical p values. They compared the relative efficacy of this approach with alternative implementations for various types and amounts of DIF, sample sizes, numbers of response categories, and amounts of impact (latent mean differences). Results indicated that the proposed strategy was considerably more effective than an alternative approach involving a constrained-baseline model. Both MACS and IRT performed similarly well in the majority of experimental conditions. As expected, MACS performed slightly worse in dichotomous conditions but better than IRT in polytomous cases where sample sizes were small. Also, contrary to popular belief, MACS performed well in conditions where DIF was simulated on item thresholds (item means), and its accuracy was not affected by impact. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessing the fit of measurement models at the individual level: A comparison of item response theory and covariance structure approaches.

The goal of this study was to explore similarities and differences in person-fit assessment under item response theory (IRT) and covariance structure analysis (CSA) measurement models. The responses of 3,245 individuals who completed 3 personality scales were analyzed under an IRT model and a CSA model. The authors then computed person-fit statistics for individual examinees under both IRT and CSA models. To be specific, for each examinee, the authors computed a standardized person-fit index for the IRT models, called Zl; in addition, an individual's contribution to chi-square, called IND{chi}, was used as a person-fit indicator for CSA models. Findings indicated that these indices are relatively free of confounds with examinee trait level. However, the relationship between Zl, and IND{chi}, values was small, suggesting that the indices identify different examinees as not fitting a model. Implications of the results and directions for future inquiry are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance.

Investigated the utility of confirmatory factor analysis (CFA) and item response theory (IRT) models for testing the comparability of psychological measurements. Both procedures were used to investigate whether mood ratings collected in Minnesota and China were comparable. Several issues were addressed. The 1st issue was that of establishing a common measurement scale across groups, which involves full or partial measurement invariance of trait indicators. It is shown that using CFA or IRT models, test items that function differentially as trait indicators across groups need not interfere with comparing examinees on the same trait dimension. Second, the issue of model fit was addressed. It is proposed that person-fit statistics be used to judge the practical fit of IRT models. Finally, topics for future research are suggested. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Looking Closer at the Effects of Framing on Risky Choice: An Item Response Theory Analysis

Item response theory (IRT) methodology allowed an in-depth examination of several issues that would be difficult to explore using traditional methodology. IRT models were estimated for 4 risky-choice items, answered by students under either a gain or loss frame. Results supported the typical framing finding of risk-aversion for gains and risk-seeking for losses but also suggested that a latent construct we label preference for risk was influential in predicting risky choice. Also, the Asian Disease item, most often used in framing research, was found to have anomalous statistical properties when compared to other framing items. Copyright 1998 Academic Press.     

Psychopathy and ethnicity: Structural, item, and test generalizability of the Psychopathy Checklist—Revised (PCL-R) in Caucasian and African American participants.

The Psychopathy Checklist--Revised (PCL-R) is an important measure in both applied and research settings. Evidence for its validity is mostly derived from male Caucasian participants. PCL-R ratings of 359 Caucasian and 356 African American participants were compared using confirmatory factor analysis (CFA) and item response theory (IRT) analyses. Previous research has indicated that 13 items of the PCL-R can be described by a 3-factor hierarchical model. This model was replicated in this sample. No cross-group difference in factor structure could be found using CFA; the structure of psychopathy is the same in both groups. IRT methods indicated significant but small differences in the performance of 5 of the 20 PCL-R items. No significant differential test functioning was found, indicating that the item differences canceled each other out. It is concluded that the PCL-R can be used, in an unbiased way, with African American participants. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Combining activities of daily living with instrumental activities of daily living to measure functional disability

Measures of functional disability typically contain items that reflect limitations in performing activities of daily living (ADLs) or instrumental activities of daily living (IADLs). Combining IADL and ADL items together in the same scale would provide enhanced range and sensitivity of measurement. This article presents psychometric justification for a combined ADL/IADL scale. Data come from 2,977 disabled respondents in the 1989 National Long-Term Care Survey. Respondents indicated whether they received human help on 7 ADL items; they also indicated whether they were unable to perform each of 9 IADL items due to health reasons. Factor analyses using tetrachoric correlations demonstrated that 15 of the 16 items reflected one major dimension. Item response theory (IRT) methods were used to calibrate the items; a one-parameter IRT model fit the data. Item calibrations showed that ADL and IADL items were not hierarchically related. Analyses showed that a simple sum of item responses could be used to derive a measure of functional disability. Implications of using a 15-item ADL/IADL scale for eligibility determination and for comparing groups are discussed.     

Identification of unique cultural response patterns by means of item response theory.

An item response theory (IRT) analysis was used to identify unique cultural response patterns by comparing single-culture groups with a multicultural composite. A survey designed to measure attitudes toward mental health was administered in their native languages to American, German, and French working, retired, and student teachers. Item characteristic curves (ICCs) for each national group were compared with ICCs generated by composite reference containing all 3 cultural groups, thus providing an omnicultural reference point. Items that exhibited differential item functioning, that is, items with dissimilar ICCs for the composite reference and focal groups, were indicative of unique cultural response patterns to the attitude survey items. The advantages and disadvantages of this method in an IRT are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analyzing Psychopathology Items: A Case for Nonparametric Item Response Theory Modeling.

The authors discuss the applicability of nonparametric item response theory (IRT) models to the construction and psychometric analysis of personality and psychopathology scales, and they contrast these models with parametric IRT models. They describe the fit of nonparametric IRT to the Depression content scale of the Minnesota Multiphasic Personality Inventory-2 (J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegen, & B. Kaemmer, 1989). They also show how nonparametric IRT models can easily be applied and how misleading results from parametric IRT models can be avoided. They recommend the use of nonparametric IRT modeling prior to using parametric logistic models when investigating personality data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Summed-score linking using item response theory: Application to depression measurement.

An item response theory (IRT) approach to test linking based on summed scores is presented and demonstrated by calibrating a modified 23-item version of the Center for Epidemiologic Studies Depression Scale (CES-D) to the standard 20-item CES-D. Data are from the Depression Patient Outcomes Research Team, 11, which used a modified CES-D to measure risk for depression. Responses (N?=?1,120) to items on both the original and modified versions were calibrated simultaneously using F. Samejima's (1969, 1997) graded IRT model. The 2 scales were linked on the basis of derived summed-score-to-IRT-score translation tables. The established cut score of 16 on the standard CES-D corresponded most closely to a summed score of 20 on the modified version. The IRT summed-score approach to test linking is a straightforward, valid, and practical method that can be applied in a variety of situations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)