Evaluating model fit for growth curve models: Integration of fit indices from SEM and MLM frameworks.

Evaluating overall model fit for growth curve models involves 3 challenging issues. (a) Three types of longitudinal data with different implications for model fit may be distinguished: balanced on time with complete data, balanced on time with data missing at random, and unbalanced on time. (b) Traditional work on fit from the structural equation modeling (SEM) perspective has focused only on the covariance structure, but growth curve models have four potential sources of misspecification: within-individual covariance matrix, between-individuals covariance matrix, marginal mean structure, and conditional mean structure. (c) Growth curve models can be estimated in both the SEM and multilevel modeling (MLM) frameworks; these have different emphases for the evaluation of model fit. In this article, the authors discuss the challenges presented by these 3 issues in the calculation and interpretation of SEM- and MLM-based fit indices for growth curve models and conclude by identifying some lines for future research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Latent variable analyses of age trends of cognition in the Health and Retirement Study, 1992-2004.

The present study was conducted to better describe age trends in cognition among older adults in the longitudinal Health and Retirement Study (HRS) from 1992 to 2004 (N = 17,000). The authors used contemporary latent variable models to organize this information in terms of both cross-sectional and longitudinal inferences about age and cognition. Common factor analysis results yielded evidence for at least 2 common factors, labeled Episodic Memory and Mental Status, largely separable from vocabulary. Latent path models with these common factors were based on demographic characteristics. Multilevel models of factorial invariance over age indicated that at least 2 common factors were needed. Latent curve models of episodic memory were based on age at testing and showed substantial age differences and age changes, including impacts due to retesting as well as several time-invariant and time-varying predictors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analyzing true change in longitudinal multitrait-multimethod studies: Application of a multimethod change model to depression and anxiety in children.

The authors show how structural equation modeling can be applied to analyze change in longitudinal multitrait-multimethod (MTMM) studies. For this purpose, an extension of latent difference models (McArdle, 1988; Steyer, Eid, & Schwenkmezger, 1997) to multiple constructs and multiple methods is presented. The model allows investigators to separate true change from measurement error and to analyze change simultaneously for different methods. The authors also show how Campbell and Fiske’s (1959) guidelines for analyzing convergent and discriminant validity can be applied to the measurement of latent change. The practical application of the multimethod change model is illustrated in a reanalysis of child depression and anxiety scores (N = 906 American children) that were assessed by self- and parent reports on three measurement occasions. The analyses revealed that (a) the convergent validity of change was low for both constructs and (b) sex was a significant predictor of self-reported, but not of parent reported, anxiety states. Finally, the authors discuss advantages and limitations and compare the model with other approaches for analyzing longitudinal MTMM data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On specifying the null model for incremental fit indices in structural equation modeling.

In structural equation modeling, incremental fit indices are based on the comparison of the fit of a substantive model to that of a null model. The standard null model yields unconstrained estimates of the variance (and mean, if included) of each manifest variable. For many models, however, the standard null model is an improper comparison model. In these cases, incremental fit index values reported automatically by structural modeling software have no interpretation and should be disregarded. The authors explain how to formulate an acceptable, modified null model, predict changes in fit index values accompanying its use, provide examples illustrating effects on fit index values when using such a model, and discuss implications for theory and practice of structural equation modeling. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Investigating early literacy and numeracy: Exploring the utility of the bifactor model.

Previous research has provided evidence for the utility of the Minneapolis Kindergarten Assessment (MKA), which is a measure of early literacy and numeracy skills. The present research was undertaken to replicate previous factorial results and evaluate the relative strength of an alternative parameterization of the measurement model, the bifactor model, which was posited to correct for anomalies found in the research literature. In addition, predictive validity evidence was ascertained to evaluate the extent to which two different factorial structures differed when making predictions about later reading and mathematics outcomes. Results suggested the bifactor model provided a useful measurement model conceptualization and also provided a strong predictive model for later reading and mathematics. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Reporting results of latent growth modeling and multilevel modeling analyses: Some recommendations for rehabilitation psychology.

Objective: There has been a general increase in interest and use of modeling techniques that treat data as nested, whether it is people nested within larger units, such as families or treatment centers, or observations nested under people. The popularity can be witnessed by noting the number of new textbooks and articles related to latent growth curve modeling and multilevel modeling. This paper discusses both of these techniques in the context of longitudinal research designs, with the main purposes of highlighting some benefits and issues related to the use of these models and outlining guidelines for reporting results from studies using multilevel modeling or latent growth modeling. Implications: These longitudinal analytic techniques can be greatly beneficial to researchers conducting rehabilitation studies, but there are several issues related to their use and reporting that need to be taken into consideration. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies.

In multilevel modeling (MLM), group-level (L2) characteristics are often measured by aggregating individual-level (L1) characteristics within each group so as to assess contextual effects (e.g., group-average effects of socioeconomic status, achievement, climate). Most previous applications have used a multilevel manifest covariate (MMC) approach, in which the observed (manifest) group mean is assumed to be perfectly reliable. This article demonstrates mathematically and with simulation results that this MMC approach can result in substantially biased estimates of contextual effects and can substantially underestimate the associated standard errors, depending on the number of L1 individuals per group, the number of groups, the intraclass correlation, the sampling ratio (the percentage of cases within each group sampled), and the nature of the data. To address this pervasive problem, the authors introduce a new multilevel latent covariate (MLC) approach that corrects for unreliability at L2 and results in unbiased estimates of L2 constructs under appropriate conditions. However, under some circumstances when the sampling ratio approaches 100%, the MMC approach provides more accurate estimates. Based on 3 simulations and 2 real-data applications, the authors evaluate the MMC and MLC approaches and suggest when researchers should most appropriately use one, the other, or a combination of both approaches. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing differences between nested covariance structure models: Power analysis and null hypotheses.

For comparing nested covariance structure models, the standard procedure is the likelihood ratio test of the difference in fit, where the null hypothesis is that the models fit identically in the population. A procedure for determining statistical power of this test is presented where effect size is based on a specified difference in overall fit of the models. A modification of the standard null hypothesis of zero difference in fit is proposed allowing for testing an interval hypothesis that the difference in fit between models is small, rather than zero. These developments are combined yielding a procedure for estimating power of a test of a null hypothesis of small difference in fit versus an alternative hypothesis of larger difference. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analytic investigation of the structure of posttraumatic stress disorder symptoms.

Converging lines of evidence have called into question the validity of conceptualizations of posttraumatic stress disorder (PTSD) based on the Diagnostic and Statistical Manual of Mental Disorders (DSM; American Psychiatric Association, 2000) and suggested alternative structural models of PTSD symptomatology. We conducted a meta-analysis of 40 PTSD studies (N = 14,827 participants across studies) that used a DSM-based measure to assess PTSD severity. We aggregated correlation matrices across studies and then applied confirmatory factor analysis to the aggregated matrices to test the fit of competing models of PTSD symptomatology that have gained support in the literature. Results indicated that both prominent 4-factor models of PTSD symptomatology yielded good model fit across subsamples of studies; however, the model comprising Intrusions, Avoidance, Hyperarousal, and Dysphoria factors appeared to fit better across studies. Results also indicated that the best fitting models were not moderated by measure or sample type. Results are discussed in the context of structural models of PTSD and implications for the diagnostic nosology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reporting practices in confirmatory factor analysis: An overview and some recommendations.

Reporting practices in 194 confirmatory factor analysis studies (1,409 factor models) published in American Psychological Association journals from 1998 to 2006 were reviewed and compared with established reporting guidelines. Three research questions were addressed: (a) how do actual reporting practices compare with published guidelines? (b) how do researchers report model fit in light of divergent perspectives on the use of ancillary fit indices (e.g., L.-T. Hu & P. M. Bentler, 1999; H. W. Marsh, K.-T., Hau, & Z. Wen, 2004)? and (c) are fit measures that support hypothesized models reported more often than fit measures that are less favorable? Results indicate some positive findings with respect to reporting practices including proposing multiple models a priori and near universal reporting of the chi-square significance test. However, many deficiencies were found such as lack of information regarding missing data and assessment of normality. Additionally, the authors found increases in reported values of some incremental fit statistics and no statistically significant evidence that researchers selectively report measures of fit that support their preferred model. Recommendations for reporting are summarized and a checklist is provided to help editors, reviewers, and authors improve reporting practices. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confirmatory Factor Analysis of the Psychopathy Checklist: Screening Version in Offenders With Axis I Disorders.

One hundred forty-nine inpatients within a maximum security psychiatric facility were assessed with the Psychopathy Checklist: Screening Version (PCL:SV; S. D. Hart, D. N. Cox, & R. D. Hare, 1995). Within the total sample, 68% had a psychotic disorder and 30% met criteria for psychopathy. Using confirmatory factor analysis, the authors tested the 2-factor PCL:SV model of psychopathy and recent 3- and 4-factor models. Results indicated good fit for each model, with the 4-factor model showing best overall fit. Structural equation modeling was used to determine which psychopathy factors predicted 6-month follow-up of inpatient aggression. The 2-, 3-, and 4-factor models, respectively, accounted for 16%.27%. and 3l% of the variance in aggression. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Strategic item selection to reduce survey length: Reduction in validity?

The practical constraints of conducting rigorous research in applied settings often limit the ability to include well-validated, but lengthy, measures in survey designs. In the current study we tested the utility of using a strategic item selection process to alleviate this issue. We evaluated the measurement and structural model fit of items selected using a strategic item selection format and assessed the predictive validity of those abbreviated scales. We also compared fit and predictive validity to the full version item sets to assess the comparability of the abbreviated scales. Results indicated that measurement models and structural models for the a priori items produced similar fit when compared with full version items across both measures. When comparing the proportion of variance explained in the criterion, there were no differences between the strategically selected items and the full versions of the scales. We conclude that the a priori selection of items based on previous research may be a useful technique for developing short measures of psychological constructs that mitigate practical, analytical, and methodological issues associated with lengthy surveys. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

"The influence of sample size, estimation methods, and model specification on goodness-of-fit assessments in structural equation models": Correction to la Du and Tanaka (1989).

Reports an error in "Influence of sample size, estimation method, and model specification on goodness-of-fit assessments in structural equation models" by Terence J. la Du and J. S. Tanaka (Journal of Applied Psychology, 1989[Aug], Vol 74[4], 625-635). Figure 2 (p. 631) summarizes Katzell's work motivation model and indicates where the trivial misspecification (dashed line) and nontrivial misspecification (starred line) occurred in our model specification condition. The error is in the latter. The starred line should be from Operations and Resources to Extrinsic Rewards and not from Rewards for Performance to Fruity. Our findings are not changed by this error, because we were using Katzell's model and accompanying data base to conduct a sampling study on goodness-of-fit indices and not testing his model. Hence, any of the paths were candidates for the nontrivial misspecification condition. (The following abstract of the original article appeared in record 1989-38703-001.) The problem of assessing fit of structural equation models is reviewed, and two sampling studies are reported that examine the effects of sample size, estimation method, and model misspecification on fit indices. In the first study, the behavior of indices in a known-population confirmatory factor analysis model is considered. In the second study, the same problem in an empirical data set is examined by looking at antecedents and consequences of work motivation. The findings across the two studies suggest that (a) as might be expected, sample size is an important determinant in assessing model fit; (b) estimator-specific, as opposed to estimator-general, fit indices provide more accurate indications of model fit; and (c) the studied fit indices are differentially sensitive to model misspecification. Some recommendations for the use of structural equation model fit indices are given. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Masking misfit in confirmatory factor analysis by increasing unique variances: A cautionary note on the usefulness of cutoff values of fit indices.

Fit indices are widely used in order to test the model fit for structural equation models. In a highly influential study, Hu and Bentler (1999) showed that certain cutoff values for these indices could be derived, which, over time, has led to the reification of these suggested thresholds as “golden rules” for establishing the fit or other aspects of structural equation models. The current study shows how differences in unique variances influence the value of the global chi-square model test and the most commonly used fit indices: Root-mean-square error of approximation, standardized root-mean-square residual, and the comparative fit index. Using data simulation, the authors illustrate how the value of the chi-square test, the root-mean-square error of approximation, and the standardized root-mean-square residual are decreased when unique variances are increased although model misspecification is present. For a broader understanding of the phenomenon, the authors used different sample sizes, number of observed variables per factor, and types of misspecification. A theoretical explanation is provided, and implications for the application of structural equation modeling are discussed. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A Discrepancy in Analyses of the MSCEIT--Resolving the Mystery and Understanding Its Implications: A Reply to Gignac (2005).

Replies to comments made by G. E. Gignac (see record 2005-06671-010) on the current authors' original article (see record 2003-02341-015). Gignac reanalyzed the factor structure of the Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) and found results that differed from those the authors obtained initially. The authors tracked down the surprising sources of those discrepancies. G. E. Gignac's hierarchical model of emotional intelligence appears promising, and the authors anticipate that further investigations of the MSCEIT factor structure may yield additional information. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Stress reactivity following brief treatment for depression: Differential effects of psychotherapy and medication.

Psychotherapy and medication treatments are both effective in reducing depressive symptoms. However, only psychotherapy provides an enduring effect by reducing depressive vulnerability following treatment termination. This differential efficacy may reflect mode-specific effects on the longitudinal relationship between depression and stress. The current study examined posttreatment data from 153 outpatients enrolled in the Treatment of Depression Collaborative Research Program. Longitudinal analyses using the latent difference score (LDS) framework (a structural modeling technique that combines features of latent growth curve and cross-lagged regression models) evaluated the temporal relationship between severity of depression and frequency of stressful life events, assessed by interviewers at treatment termination and at 6, 12, and 18 months following treatment. Results supported a stress reactivity model in that stressful events led to elevations in the rate of depression change. Furthermore, multigroup LDS analysis indicated that this longitudinal stress reactivity occurred only for outpatients in the medication conditions. Results demonstrate that the enduring impact of psychotherapy involves the development of enhanced resiliency to stressful life events. (PsycINFO Database Record (c) 2010 APA, all rights reserved)