Pooling data from multiple longitudinal studies: The role of item response theory in integrative data analysis.

There are a number of significant challenges researchers encounter when studying development over an extended period of time, including subject attrition, the changing of measurement structures across groups and developmental periods, and the need to invest substantial time and money. Integrative data analysis is an emerging set of methodologies that allows researchers to overcome many of the challenges of single-sample designs through the pooling of data drawn from multiple existing developmental studies. This approach is characterized by a host of advantages, but this also introduces several new complexities that must be addressed prior to broad adoption by developmental researchers. In this article, the authors focus on methods for fitting measurement models and creating scale scores using data drawn from multiple longitudinal studies. The authors present findings from the analysis of repeated measures of internalizing symptomatology that were pooled from three existing developmental studies. The authors describe and demonstrate each step in the analysis and conclude with a discussion of potential limitations and directions for future research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Integrative data analysis: The simultaneous analysis of multiple data sets.

There are both quantitative and methodological techniques that foster the development and maintenance of a cumulative knowledge base within the psychological sciences. Most noteworthy of these techniques is meta-analysis, which allows for the synthesis of summary statistics drawn from multiple studies when the original data are not available. However, when the original data can be obtained from multiple studies, many advantages stem from the statistical analysis of the pooled data. The authors define integrative data analysis (IDA) as the analysis of multiple data sets that have been pooled into one. Although variants of IDA have been incorporated into other scientific disciplines, the use of these techniques is much less evident in psychology. In this article the authors present an overview of IDA as it may be applied within the psychological sciences, discuss the relative advantages and disadvantages of IDA, describe analytic strategies for analyzing pooled individual data, and offer recommendations for the use of IDA in practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models.

[Correction Notice: An erratum for this article was reported in Vol 14(4) of Psychological Methods (see record 2009-22665-007). In this article, the authors wrote, "To our knowledge, the multisample framework is the only available option within these [latent variable] programs that allows for the moderation of all types of parameters, and this approach requires a single categorical moderator variable to define the samples.” Bengt Muthén has clarified for the authors that some programs, including Mplus and Mx, can allow for continuous moderation through the implementation of nonlinear constraints involving observed variables, further enlarging the class of MNLFA models that can be fit with these programs.] When conducting an integrative analysis of data obtained from multiple independent studies, a fundamental problem is to establish commensurate measures for the constructs of interest. Fortunately, procedures for evaluating and establishing measurement equivalence across samples are well developed for the linear factor model and commonly used item response theory models. A newly proposed moderated nonlinear factor analysis model generalizes these models and procedures, allowing for items of different scale types (continuous or discrete) and differential item functioning across levels of categorical and/or continuous variables. The potential of this new model to resolve the problem of measurement in integrative data analysis is shown via an empirical example examining changes in alcohol involvement from ages 10 to 22 years across 2 longitudinal studies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Short and long views of integrative data analysis: Comments on contributions to the special issue.

The author comments on the potential of integrative data analysis (IDA) as a new methodological activity and on some of the topics that were discussed in the 5 articles in this special issue. One topic is the extent to which IDA will be used to provide conclusive summaries regarding the strength of evidence for well-specified questions versus to provide new information that goes beyond the simple sum of individual studies. Another is the meaning of variances of effects that are observed over studies and sample strata. A 3rd is the potential to enhance understanding of construct validity by fitting measurement models described in the special issue. The author concludes by recommending critical examination of model-based inferences from IDA through sensitivity analyses and by noting that IDA can promote collaboration and networks that yield data that are more amenable to integrated analyses in the future. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models": Clarification to Bauer and Hussong (2009).

Reports a clarification to "Psychometric approaches for developing commensurate measures across independent studies: Traditional and new models" by Daniel J. Bauer and Andrea M. Hussong (Psychological Methods, 2009[Jun], Vol 14[2], 101-125). In this article, the authors wrote, "To our knowledge, the multisample framework is the only available option within these [latent variable] programs that allows for the moderation of all types of parameters, and this approach requires a single categorical moderator variable to define the samples.” Bengt Muthén has clarified for the authors that some programs, including Mplus and Mx, can allow for continuous moderation through the implementation of nonlinear constraints involving observed variables, further enlarging the class of MNLFA models that can be fit with these programs. (The following abstract of the original article appeared in record 2009-08072-001.) When conducting an integrative analysis of data obtained from multiple independent studies, a fundamental problem is to establish commensurate measures for the constructs of interest. Fortunately, procedures for evaluating and establishing measurement equivalence across samples are well developed for the linear factor model and commonly used item response theory models. A newly proposed moderated nonlinear factor analysis model generalizes these models and procedures, allowing for items of different scale types (continuous or discrete) and differential item functioning across levels of categorical and/or continuous variables. The potential of this new model to resolve the problem of measurement in integrative data analysis is shown via an empirical example examining changes in alcohol involvement from ages 10 to 22 years across 2 longitudinal studies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Use of missing data methods in longitudinal studies: The persistence of bad practices in developmental psychology.

Developmental science rests on describing, explaining, and optimizing intraindividual changes and, hence, empirically requires longitudinal research. Problems of missing data arise in most longitudinal studies, thus creating challenges for interpreting the substance and structure of intraindividual change. Using a sample of reports of longitudinal studies obtained from three flagship developmental journals—Child Development, Developmental Psychology, and Journal of Research on Adolescence—we examined the number of longitudinal studies reporting missing data and the missing data techniques used. Of the 100 longitudinal studies sampled, 57 either reported having missing data or had discrepancies in sample sizes reported for different analyses. The majority of these studies (82%) used missing data techniques that are statistically problematic (either listwise deletion or pairwise deletion) and not among the methods recommended by statisticians (i.e., the direct maximum likelihood method and the multiple imputation method). Implications of these results for developmental theory and application, and the need for understanding the consequences of using statistically inappropriate missing data techniques with actual longitudinal data sets, are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Modeling life-span growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement.

The authors use multiple-sample longitudinal data from different test batteries to examine propositions about changes in constructs over the life span. The data come from 3 classic studies on intellectual abilities in which, in combination, 441 persons were repeatedly measured as many as 16 times over 70 years. They measured cognitive constructs of vocabulary and memory using 8 age-appropriate intelligence test batteries and explore possible linkage of these scales using item response theory (IRT). They simultaneously estimated the parameters of both IRT and latent curve models based on a joint model likelihood approach (i.e., NLMIXED and WINBUGS). They included group differences in the model to examine potential interindividual differences in levels and change. The resulting longitudinal invariant Rasch test analyses lead to a few new methodological suggestions for dealing with repeated constructs based on changing measurements in developmental studies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Mean-level change in self-esteem from childhood through adulthood: Meta-analysis of longitudinal studies.

This study performs a meta-analysis of the mean-level of change in self-esteem across the life span. Fifty-nine studies yielded data from 130 independent samples. Results indicate that, despite slightly increasing from childhood to the first decade of young adulthood, self-esteem does not change beyond 30 years old. Self-esteem changes the most during the first decade of young adulthood. The effects of gender and time span between assessments on change in self-esteem were minimal during adolescence, while the way self-esteem is measured significantly affects change. The mean effect size was the largest with the Coopersmith Self-Esteem Inventory and the smallest with Harter's Self-Perception Profile. Birth cohort also significantly influences change, whereas older cohorts change to a smaller extent compared to younger cohorts. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies.

The present study used meta-analytic techniques (number of samples = 92) to determine the patterns of mean-level change in personality traits across the life course. Results showed that people increase in measures of social dominance (a facet of extraversion), conscientiousness, and emotional stability, especially in young adulthood (age 20 to 40). In contrast, people increase on measures of social vitality (a 2nd facet of extraversion) and openness in adolescence but then decrease in both of these domains in old age. Agreeableness changed only in old age. Of the 6 trait categories, 4 demonstrated significant change in middle and old age. Gender and attrition had minimal effects on change, whereas longer studies and studies based on younger cohorts showed greater change. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The analysis of profile data.

"Methods were suggested for handling 3 problems in the analysis of test profiles: measuring the similarity of profiles, discriminating the typical profiles of two or more groups, and clustering profiles into homogeneous groups. The suggested methods were, respectively, picturing profiles as interpoint distances in Euclidean space, use of the linear multiple-discriminant function, and factor analysis of profile cross-product terms. Some suggestions were given about transformations of profile data before further analysis." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analyzing longitudinal data with multilevel models: An example with individuals living with lower extremity intra-articular fractures.

Objective: The use and quality of longitudinal research designs has increased over the past 2 decades, and new approaches for analyzing longitudinal data, including multilevel modeling (MLM) and latent growth modeling (LGM), have been developed. The purpose of this article is to demonstrate the use of MLM and its advantages in analyzing longitudinal data. Research Method: Data from a sample of individuals with intra-articular fractures of the lower extremity from the University of Alabama at Birmingham's Injury Control Research Center are analyzed using both SAS PROC MIXED and SPSS MIXED. Results: The authors begin their presentation with a discussion of data preparation for MLM analyses. The authors then provide example analyses of different growth models, including a simple linear growth model and a model with a time-invariant covariate, with interpretation for all the parameters in the models. Implications: More complicated growth models with different between- and within-individual covariance structures and nonlinear models are discussed. Finally, information related to MLM analysis, such as online resources, is provided at the end of the article. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An array of qualitative data analysis tools: A call for data analysis triangulation.

One of the most important steps in the qualitative research process is analysis of data. The purpose of this article is to provide elements for understanding multiple types of qualitative data analysis techniques available and the importance of utilizing more than one type of analysis, thus utilizing data analysis triangulation, in order to understand phenomenon more fully for school psychology research and beyond. The authors describe seven qualitative analysis tools: methods of constant comparison, keywords-in-context, word count, classical content analysis, domain analysis, taxonomic analysis, and componential analysis. Then, the authors outline when to use each type of analysis. In so doing, the authors use real qualitative data to help distinguish the various types of analyses. Furthermore, flowcharts and tables are provided to help delineate when to choose each type of analysis. Finally, the role of computer-assisted software in the qualitative data-analytic process is discussed. As such, use of the analyses outlined in this article should help to promote rigor in qualitative research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A cautionary note on modeling growth trends in longitudinal data.

Random coefficient and latent growth curve modeling are currently the dominant approaches to the analysis of longitudinal data in psychology. The application of these models to longitudinal data assumes that the data-generating mechanism behind the psychological process under investigation contains only a deterministic trend. However, if a process, at least partially, contains a stochastic trend, then random coefficient regression results are likely to be spurious. This problem is demonstrated via a data example, previous research on simple regression models, and Monte Carlo simulations. A data analytic strategy is proposed to help researchers avoid making inaccurate inferences when observed trends may be due to stochastic processes. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The seemingly quixotic pursuit of a cumulative psychological science: Introduction to the special issue.

The goal of any empirical science is to pursue the construction of a cumulative base of knowledge upon which the future of the science may be built. However, there is mixed evidence that the science of psychology can accurately be characterized by such a cumulative progression. Indeed, some argue that the development of a truly cumulative psychological science is not possible with the current paradigms of hypothesis testing in single-study designs. The author explores this controversy as a framework to introduce the 6 articles that make up this special issue on the integration of data and empirical findings across multiple studies. The author proposes that the methods and techniques described in this set of articles can significantly propel researchers forward in their ongoing quest to build a cumulative psychological science. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rumination, emotion, and forgiveness: Three longitudinal studies.

In 3 studies, the authors investigated whether within-persons increases in rumination about an interpersonal transgression were associated with within-persons reductions in forgiveness. Results supported this hypothesis. The association of transient increases in rumination with transient reductions in forgiveness appeared to be mediated by anger, but not fear, toward the transgressor. The association of rumination and forgiveness was not confounded by daily fluctuations in positive affect and negative affect, and it was not moderated by trait levels of positive affectivity, negative affectivity, or perceived hurtfulness of the transgression. Cross-lagged associations of rumination and forgiveness in Study 3 more consistently supported the proposition that increased rumination precedes reductions in forgiveness than the proposition that increased forgiveness precedes reductions in rumination. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions.

This article demonstrates assumptions of invariance that researchers often implicitly make when analyzing multilevel data. The first set of assumptions is measurement-based and corresponds to the fact that researchers often conduct single-level exploratory and confirmatory factor analyses, and reliability analyses, with multilevel data. The second assumption, that of structural invariance, is engineered into the common multilevel random coefficient model, in that such analyses impose structural invariance across multiple levels of analysis when lower-level relationships represent both between- and within-groups effects. The nature of these assumptions, and ways to address their tenability, are explored from a conceptual standpoint. Then an empirical example of these assumptions and ways to address them is provided. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Application of canonical correlational analysis to learning data.

Considers the application of Hotelling's canonical correlation analysis to certain problems of learning, such as (1) prediction of learning from external measures, (2) efficiency of learning indices as predictors of academic grades, (3) the extent to which different sets of learning scores share the same function, and (4) changes in the factorial structure of learning as practice continues. Analyses of the published data using this statistical method reveal that there is a considerable amount of improvement in predictive efficiency if learning is treated in multivariate terms. An important methodological point is the finding that in classical eyelidconditioning experiments, the Ss should be matched in terms of their reflex sensitivity to light and puff. It is also felt that canonical analysis may serve as an alternative method of studying the nature and extent of change in ability patterns as improvement occurs in a learning task. (51 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The proof of the pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis.

Some methodologists have recently suggested that scientific psychology's overreliance on null hypothesis significance testing (NHST) impedes the progress of the discipline. In response, a number of defenders have maintained that NHST continues to play a vital role in psychological research. Both sides of the argument to date have been presented abstractly. The authors take a different approach to this issue by illustrating the use of NHST along with 2 possible alternatives (meta-analysis as a primary data analysis strategy and Bayesian approaches) in a series of 3 studies. Comparing and contrasting the approaches on actual data brings out the strengths and weaknesses of each approach. The exercise demonstrates that the approaches are not mutually exclusive but instead can be used to complement one another. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

All data collection and analysis methods have limitations: Reply to Rabbitt (2011) and Raz and Lindenberger (2011).

The commentaries on my article contain a number of points with which I disagree but also several with which I agree. For example, I continue to believe that the existence of many cases in which between-person variability does not increase with age indicates that greater variance with increased age is not inevitable among healthy individuals up to about 80 years of age. I also do not believe that problems of causal inferences from correlational information are more severe in the cognitive neuroscience of aging than in other research areas; I contend instead that neglect of these problems has led to confusion about neurobiological underpinnings of cognitive aging. I agree that researchers need to be cautious in extrapolating from cross-sectional to longitudinal relations, but I also note that even longitudinal data are limited with respect to their ability to support causal inferences. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Beyond constant comparison qualitative data analysis: Using NVivo.

The purposes of this paper are to outline seven types of qualitative data analysis techniques, to present step-by-step guidance for conducting these analyses via a computer-assisted qualitative data analysis software program (i.e., NVivo9), and to present screenshots of the data analysis process. Specifically, the following seven analyses are presented: constant comparison analysis, classical content analysis, keyword-in-context, word count, domain analysis, taxonomic analysis, and componential analysis. It is our hope that providing a clear step-by-step process for conducting these analyses with NVivo9 will assist school psychology researchers in increasing the rigor of their qualitative data analysis procedures. (PsycINFO Database Record (c) 2011 APA, all rights reserved)