Item factor analysis: Current approaches and future directions.

The rationale underlying factor analysis applies to continuous and categorical variables alike; however, the models and estimation methods for continuous (i.e., interval or ratio scale) data are not appropriate for item-level data that are categorical in nature. The authors provide a targeted review and synthesis of the item factor analysis (IFA) estimation literature for ordered-categorical data (e.g., Likert-type response scales) with specific attention paid to the problems of estimating models with many items and many factors. Popular IFA models and estimation methods found in the structural equation modeling and item response theory literatures are presented. Following this presentation, recent developments in the estimation of IFA parameters (e.g., Markov chain Monte Carlo) are discussed. The authors conclude with considerations for future research on IFA, simulated examples, and advice for applied researchers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychometric properties of the MMPI-2-RF Somatic Complaints (RC1) Scale.

The MMPI-2 Restructured Form (MMPI-2-RF; Tellegen & Ben-Porath, 2008) was designed to be psychometrically superior to its MMPI-2 counterpart. However, the test has yet to be extensively evaluated in diverse clinical settings. The purpose of this study was to examine the psychometric properties of the MMPI-2-RF Somatic Complaints (RC1) scale in a clinically relevant population. Participants were 399 patients diagnosed with either epilepsy or psychogenic nonepileptic seizures on the basis of video–electroencephalograph monitoring. The internal structure of the MMPI-2-RF was evaluated using taxometric, confirmatory factor analysis, and item response theory procedures. Data from 4 content-specific scales directly related to RC1 (Malaise, Gastrointestinal Complaints, Head Pain Complaints, and Neurological Complaints) indicated that the latent construct of somatization is a dimensional variable with a bifactor structure. However, consistent with the scale's construction, a unidimensional model also provided adequate fit. A 2-parameter logistic item response theory model better accounted for observed item responses than did 1- or 3-parameter models. Results suggest that the RC1 scale is most precise for T score estimates between 55 and 90. Overall, the scale appears to be well suited for the assessment of somatization. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effect of ignoring item interactions on the estimated discrimination parameters in item response theory.

Most item response theory models assume conditional independence, and it is known that interactions between items affect the estimated item discrimination. In this article, this effect is further investigated from a theoretical perspective and by means of simulation studies. To this end, a parametric model for item interactions is introduced. Next, it is shown that ignoring a positive interaction results in an overestimation of the discrimination parameter in the two-parameter logistic model (2PLM), whereas ignoring a negative interaction leads to an underestimation of the parameter. Furthermore, it is demonstrated that in some cases the item characteristic curves of the 2PLM and of an item involved in an interaction are quite similar, indicating that the 2PLM can provide a good fit to data with interactions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Bifactor and item response theory analyses of interviewer report scales of cognitive impairment in schizophrenia.

A psychometric analysis of 2 interview-based measures of cognitive deficits was conducted: the 21-item Clinical Global Impression of Cognition in Schizophrenia (CGI-CogS; Ventura et al., 2008), and the 20-item Schizophrenia Cognition Rating Scale (SCoRS; Keefe et al., 2006), which were administered on 2 occasions to a sample of people with schizophrenia. Traditional psychometrics, bifactor analysis, and item response theory methods were used to explore item functioning and dimensionality and to compare instruments. Despite containing similar item content, responses to the CGI-CogS demonstrated superior psychometric properties (e.g., higher item intercorrelations, better spread of ratings across response categories) relative to the SCoRS. The authors argue that these differences arise mainly from the differential use of prompts and how the items are phrased and scored. Bifactor analysis demonstrated that although both measures capture a broad range of cognitive functioning (e.g., working memory, social cognition), the common variance on each is overwhelmingly explained by a single general factor. Item response theory analyses of the combined pool of 41 items showed that measurement precision is peaked in the mild to moderate range of cognitive impairment. Finally, simulated adaptive testing revealed that only about 10 to 12 items are necessary to achieve latent trait level estimates with reasonably small standard errors for most individuals. This suggests that these interview-based measures of cognitive deficits could be shortened without loss of measurement precision. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences.

This article analyzes latent variable models from a cognitive psychology perspective. We start by discussing work by Tuerlinckx and De Boeck (2005), who proved that a diffusion model for 2-choice response processes entails a 2-parameter logistic item response theory (IRT) model for individual differences in the response data. Following this line of reasoning, we discuss the appropriateness of IRT for measuring abilities and bipolar traits, such as pro versus contra attitudes. Surprisingly, if a diffusion model underlies the response processes, IRT models are appropriate for bipolar traits but not for ability tests. A reconsideration of the concept of ability that is appropriate for such situations leads to a new item response model for accuracy and speed based on the idea that ability has a natural zero point. The model implies fundamentally new ways to think about guessing, response speed, and person fit in IRT. We discuss the relation between this model and existing models as well as implications for psychology and psychometrics. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Examining assumptions about item responding in personality assessment: Should ideal point methods be considered for scale development and scoring?

The present study investigated whether the assumptions of an ideal point response process, similar in spirit to Thurstone's work in the context of attitude measurement, can provide viable alternatives to the traditionally used dominance assumptions for personality item calibration and scoring. Item response theory methods were used to compare the fit of 2 ideal point and 2 dominance models with data from the 5th edition of the Sixteen Personality Factor Questionnaire (S. Conn & M. L. Rieke, 1994). The authors' results indicate that ideal point models can provide as good or better fit to personality items than do dominance models because they can fit monotonically increasing item response functions but do not require this property. Several implications of these findings for personality measurement and personnel selection are described. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory.

Current interest in the assessment of measurement equivalence emphasizes 2 major methods of analysis. The authors offer a comparison of a linear method (confirmatory factor analysis) and a nonlinear method (differential item and test functioning using item response theory) with an emphasis on their methodological similarities and differences. The 2 approaches test for the equality of true scores (or expected raw scores) across 2 populations when the latent (or factor) score is held constant. Both approaches can provide information about when measurrment nonequivalence exists and the extent to which it is a problem. An empirical example is used to illustrate the 2 approaches. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Diagnosing item score patterns on a test using item response theory-based person-fit statistics.

Person-fit statistics have been proposed to investigate the fit of an item score pattern to an item response theory (IRT) model. The author investigated how these statistics can be used to detect different types of misfit. Intelligence test data were analyzed using person-fit statistics in the context of the G. Rasch (1960) model and R. J. Mokken's (1971, 1997) IRT models. The effect of the choice of an IRT model to detect misfitting item score patterns and the usefulness of person-fit statistics for diagnosis of misfit are discussed. Results showed that different types of person-fit statistics can be used to detect different kinds of person misfit. Parametric person-fit statistics had more power than nonparametric person-fit statistics. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Random intercept item factor analysis.

The common factor model assumes that the linear coefficients (intercepts and factor loadings) linking the observed variables to the latent factors are fixed coefficients (i.e., common for all participants). When the observed variables are participants' observed responses to stimuli, such as their responses to the items of a questionnaire, the assumption of common linear coefficients may be too restrictive. For instance, this may occur if participants consistently use the response scale idiosyncratically. To account for this phenomenon, the authors partially relax the fixed coefficients assumption by allowing the intercepts in the factor model to change across participants. The model is attractive when m factors are expected on the basis of substantive theory but m + 1 factors are needed in practice to adequately reproduce the data. Also, this model for single-level data can be fitted with conventional software for structural equation modeling. The authors demonstrate the use of this model with an empirical data set on optimism in which they compare it with competing models such as the bifactor and the correlated trait-correlated method minus 1 models. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychopathy and ethnicity: Structural, item, and test generalizability of the Psychopathy Checklist—Revised (PCL-R) in Caucasian and African American participants.

The Psychopathy Checklist--Revised (PCL-R) is an important measure in both applied and research settings. Evidence for its validity is mostly derived from male Caucasian participants. PCL-R ratings of 359 Caucasian and 356 African American participants were compared using confirmatory factor analysis (CFA) and item response theory (IRT) analyses. Previous research has indicated that 13 items of the PCL-R can be described by a 3-factor hierarchical model. This model was replicated in this sample. No cross-group difference in factor structure could be found using CFA; the structure of psychopathy is the same in both groups. IRT methods indicated significant but small differences in the performance of 5 of the 20 PCL-R items. No significant differential test functioning was found, indicating that the item differences canceled each other out. It is concluded that the PCL-R can be used, in an unbiased way, with African American participants. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improving Measurement Precision of Test Batteries Using Multidimensional Item Response Models.

A conventional way to analyze item responses in multiple tests is to apply unidimensional item response models separately, one test at a time. This unidimensional approach, which ignores the correlations between latent traits, yields imprecise measures when tests are short. To resolve this problem, one can use multidimensional item response models that use correlations between latent traits to improve measurement precision of individual latent traits. The improvements are demonstrated using 2 empirical examples. It appears that the multidimensional approach improves measurement precision substantially, especially when tests are short and the number of tests is large. To achieve the same measurement precision, the multidimensional approach needs less than half of the comparable items required for the unidimensional approach. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of estimation methods on making trait-level inferences from ordered categorical items for assessing psychopathology.

In assessments of attitudes, personality, and psychopathology, unidimensional scale scores are commonly obtained from Likert scale items to make inferences about individuals' trait levels. This study approached the issue of how best to combine Likert scale items to estimate test scores from the practitioner's perspective: Does it really matter which method is used to estimate a trait? Analyses of 3 data sets indicated that commonly used methods could be classified into 2 groups: methods that explicitly take account of the ordered categorical item distributions (i.e., partial credit and graded response models of item response theory, factor analysis using an asymptotically distribution-free estimator) and methods that do not distinguish Likert-type items from continuously distributed items (i.e., total score, principal component analysis, maximum-likelihood factor analysis). Differences in trait estimates were found to be trivial within each group. Yet the results suggested that inferences about individuals' trait levels differ considerably between the 2 groups. One should therefore choose a method that explicitly takes account of item distributions in estimating unidimensional traits from ordered categorical response formats. Consequences of violating distributional assumptions were discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of two methods for estimating item response theory parameters when assessing differential item functioning.

Recent legal developments appear to sanction the use of psychometrically unsound procedures for examining differential item functioning (DIF) on standardized tests. More appropriate approaches involve the use of item response theory (IRT). However, many IRT-based DIF studies have used F. M. Lord's (see record 1987-17535-001) joint maximum likelihood procedure, which can lead to incorrect and misleading results. A Monte Carlo simulation was conducted to evaluate the effectiveness of two other methods of parameter estimation: marginal maximum likelihood estimation and Bayes modal estimation. Sample size and data dimensionality were manipulated in the simulation. Results indicated that both estimation methods (a) provided more accurate parameter estimates and less inflated Type I error rates than joint maximum likelihood, (b) were robust to multidimensionality, and (c) produced more accurate parameter estimates and higher rates of identifying DIF with larger samples. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The construction of unidimensional tests.

"Five methods of constructing unidimensional tests (classical item analysis, Loevinger's procedure, the independence criterion method, the answer patterns method, and factor analysis) have been considered with respect to their provision for: a rational procedure for item selection, a criterion of unidimensionality, and an index of unidimensionality." Only factor analysis provides a rational procedure for item selection. No method has a fully satisfactory criterion of unidimensionality. The ratio of 1st to 2nd factor variance has been suggested as an index of unidimensionality for the factor analytic method; but it suffers from a lack of knowledge of its sampling fluctuations, a weakness shared with the other procedures. From Psyc Abstracts 36:02:2AH22L. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confirmatory factor analysis of items with a dichotomous response format using the multidimensional Rasch model.

Two models that can be used for exploratory factor analysis of items with a dichotomous response format are discussed: threshold models and multidimensional item response models. The models arise from different traditions: The threshold model is rooted in the factor analytic tradition, the multidimensional item response model had its foundation in item response theory. Despite the different origins, it can be proved that both models are the same. Subsequently, the generalized multidimensional Rasch model is introduced. This model can be used for confirmatory factor analysis of items with a dichotomous response format. Stated otherwise, it is the confirmatory counterpart of the (exploratory) threshold and multidimensional item response models. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models.

[Correction Notice: An erratum for this article was reported in Vol 14(4) of Psychological Methods (see record 2009-22665-007). In this article, the authors wrote, "To our knowledge, the multisample framework is the only available option within these [latent variable] programs that allows for the moderation of all types of parameters, and this approach requires a single categorical moderator variable to define the samples.” Bengt Muthén has clarified for the authors that some programs, including Mplus and Mx, can allow for continuous moderation through the implementation of nonlinear constraints involving observed variables, further enlarging the class of MNLFA models that can be fit with these programs.] When conducting an integrative analysis of data obtained from multiple independent studies, a fundamental problem is to establish commensurate measures for the constructs of interest. Fortunately, procedures for evaluating and establishing measurement equivalence across samples are well developed for the linear factor model and commonly used item response theory models. A newly proposed moderated nonlinear factor analysis model generalizes these models and procedures, allowing for items of different scale types (continuous or discrete) and differential item functioning across levels of categorical and/or continuous variables. The potential of this new model to resolve the problem of measurement in integrative data analysis is shown via an empirical example examining changes in alcohol involvement from ages 10 to 22 years across 2 longitudinal studies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy.

In this article, the authors developed a common strategy for identifying differential item functioning (DIF) items that can be implemented in both the mean and covariance structures method (MACS) and item response theory (IRT). They proposed examining the loadings (discrimination) and the intercept (location) parameters simultaneously using the likelihood ratio test with a free-baseline model and Bonferroni corrected critical p values. They compared the relative efficacy of this approach with alternative implementations for various types and amounts of DIF, sample sizes, numbers of response categories, and amounts of impact (latent mean differences). Results indicated that the proposed strategy was considerably more effective than an alternative approach involving a constrained-baseline model. Both MACS and IRT performed similarly well in the majority of experimental conditions. As expected, MACS performed slightly worse in dichotomous conditions but better than IRT in polytomous cases where sample sizes were small. Also, contrary to popular belief, MACS performed well in conditions where DIF was simulated on item thresholds (item means), and its accuracy was not affected by impact. (PsycINFO Database Record (c) 2010 APA, all rights reserved)