Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis.

Assessment centers are widely believed to have relatively small standardized subgroup differences (d). However, no meta-analytic review to date has examined ds for assessment centers. The authors conducted a meta-analysis of available data and found an overall Black-White d of 0.52, an overall Hispanic-White d of 0.28, and an overall male-female d of -0.19. Consistent with our expectations, results suggest that Black-White ds in assessment center data may be larger than was previously thought. Hispanic-White comparisons were smaller than were Black-White comparisons. Females, on average, scored higher than did males in assessment centers. As such, assessment centers may be associated with more adverse impact against Blacks than is portrayed in the literature, but the predictor may have less adverse impact and be more "diversity friendly" for Hispanics and females. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A factor analysis of managers' ratings.

Assignment ratings on 97 insurance company division managers, for a list of 20 assignments, were factor-analyzed into an orthogonal solution containing a general factor and five group factors. The group factors were named: "Skill in dealing with others," "Judgment," "Effectiveness in supervising the work," "Effectiveness in planning the work," and "Effectiveness in improving operating efficiency." The general factor accounted for more of the variance than any of the group factors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A confirmatory maximum likelihood factor analysis of the WAIS-R.

A simultaneous maximum likelihood confirmatory factor analysis of intercorrelations among the 11 subtests of the WAIS–R was undertaken in the 9 age groups in the normative sample (D. Wechsler, 1981). Several models were fit to the data and compared, including a single-factor model; orthogonal and oblique 2-factor models, comprised of the Verbal and Performance subtests, respectively; an orthogonal and 2 oblique 3-factor models, the 3rd factor involving the Digit Span, Arithmetic, and Digit Symbol subtests; a 3-factor model, composed of a general intellectual factor and Verbal and Performance factors; and a 4-factor model, consisting of both a general factor and 3 common ability factors (Verbal Comprehension, Perceptual Organization, and Memory/Freedom from Distractability). Results suggest that (a) the single-factor model fits the data fairly well, although there is some misspecification in this model; (b) none of the multifactor models fit the data markedly better than the single-factor model; and (c) the single-factor structure is consistent across the age groups. It is concluded that the WAIS–R largely contains a general intellectual factor and that any common ability factors therein play a minor role in influencing WAIS–R subscale scores. (23 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Adherence in childhood diabetes: Results of a confirmatory factor analysis.

Tested the equality of the factor pattern of adherence to treatment of childhood diabetes identified by S. B. Johnson et al (see record 1988-05437-001) across 2 independent samples. Samples of 162 and 157 diabetics (aged 6–18 yrs) were studied. Factors 1–4, Exercise, Injection, Diet Type, and Eating/Testing Frequency, were confirmed. Factor 5, Diet Amount, proved to be too complex; the adherence measures comprising this factor (total calories and concentrated sweets consumed) are best treated as separate, single-indicator constructs. Results support a multivariate concept of adherence, offer insight into the nature of the components underlying diabetes adherence, and provide measurement information for reliable component estimation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Marital conventionalization as a measure of marital satisfaction: A confirmatory factor analysis.

It has been suggested that self-report measures of marital satisfaction are contaminated by a social desirability bias, which V. H. Edmonds (1967) termed marital conventionalization. Subsequent research showed that marital satisfaction measures were strongly correlated with marital conventionalization but not with social desirability scales. Because these measures tend to be so strongly correlated, this study examined whether marital satisfaction and conventionalization are best seen as 2 separate constructs or as a single construct. These measures were found to have 1 underlying factor based on confirmatory factor analyses with a sample of 2,109 individuals. Further analysis suggested that the relationship between marital satisfaction and conventionalization scales differs depending on the level of marital satisfaction. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Utility of confirmatory factor analysis in test validation research.

The multitrait–multimethod ({mtmm}) design is often used in test validation research to disentangle problems due to shared method variance. However, {mtmm} research requires extensive data collection that may be prohibitive in clinical settings. Furthermore, interpretation of {mtmm} data can be ambiguous and misleading. In the current article, maximum-likelihood confirmatory factor analysis ({cfa}) is presented as a means for less ambiguous interpretation of complete and incomplete {mtmm} designs. Confirmatory Factor analysis is applied to four data sets that represent four designs: monotrait–monomethod, multitrait–monomethod, monotrait–multimethod, and multitrait–multimethod. In all four cases, {cfa} results provided more rigorous support of the original authors' positions and provided valuable supplemental findings as well. Additionally, {cfa} was applied to an artificial data set. Intuitive interpretations of the artificial data were compared with the results of the {cfa}. The results showed that the intuitive approach can lead to highly spurious conclusions regarding convergent and discriminant validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The factor structure of Union Commitment: An application of confirmatory factor analysis.

M. E. Gordon et al. (see record 1980-33572-001) developed a Union Commitment scale and identified four underlying factors. L. Friedman and R. J. Harvey (see record 1986-28952-001), in a reanalysis of the Gordon et al. data, concluded that union commitment was best represented by two factors. Although several investigations have replicated the Gordon et al. factor structure, none of these have used confirmatory factor analysis (CFA). In the present study we used CFA to test the explanatory power of the two a priori factor structures. A sample of 465 blue-collar workers working for a large utility company in the midwestern United States participated in this investigation. The results suggest that, first, a modified version of Gordon et al.'s four-factor solution provides a better representation of the factor structure of union commitment. Second, the use of either the four-factor solution or commitment as a unidimensional construct may be justified, depending on the research focus. Finally, we found that unit weighting of the union commitment items was the most appropriate strategy for generating composites. The implication of these results and suggestions for future research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reporting practices in confirmatory factor analysis: An overview and some recommendations.

Reporting practices in 194 confirmatory factor analysis studies (1,409 factor models) published in American Psychological Association journals from 1998 to 2006 were reviewed and compared with established reporting guidelines. Three research questions were addressed: (a) how do actual reporting practices compare with published guidelines? (b) how do researchers report model fit in light of divergent perspectives on the use of ancillary fit indices (e.g., L.-T. Hu & P. M. Bentler, 1999; H. W. Marsh, K.-T., Hau, & Z. Wen, 2004)? and (c) are fit measures that support hypothesized models reported more often than fit measures that are less favorable? Results indicate some positive findings with respect to reporting practices including proposing multiple models a priori and near universal reporting of the chi-square significance test. However, many deficiencies were found such as lack of information regarding missing data and assessment of normality. Additionally, the authors found increases in reported values of some incremental fit statistics and no statistically significant evidence that researchers selectively report measures of fit that support their preferred model. Recommendations for reporting are summarized and a checklist is provided to help editors, reviewers, and authors improve reporting practices. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size.

In confirmatory factor analysis, hypothesized models reflect approximations to reality so that any model can be rejected if the sample size is large enough. In the present article, we examine the influence of sample size on different fit indexes for both real and simulated data. Contrary to claims by Bentler and Bonett (1980), their incremental fit index was substantially affected by sample size. Contrary to claims by Joreskog and Sorbom (1981), their goodness-of-fit indexes provided by {lisrel} were substantially affected by sample size. Contrary to claims by Bollen (1986), his new incremental fit index was substantially affected by sample size. Hoelter's (1983) critical N index was also substantially affected by sample size. Of the more than 30 indexes considered, the Tucker-Lewis (1973) index was the only widely used index that was relatively independent of sample size. However, four new indexes based on the same form as the Tucker-Lewis index were also relatively independent of sample size. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rationally and empirically derived dimensions of children's symptomatology: Expert ratings and confirmatory factor analyses of the CBCL.

Expert ratings and confirmatory factor analyses were used to develop an alternative system for scoring the Child Behavior Checklist (CBCL; T. M. Achenbach, 1991) to measure specific dimensions corresponding to current conceptualizations of child symptomatology. Data were from a nonclinic and 2 independent clinic samples. Subscales measuring Anxiety, Attention Problems/Hyperactivity, Conduct Problems, Depression, Oppositional Defiant, Social Problems/Immaturity, and Somatization were created. Logistic regressions were conducted to evaluate the diagnostic efficiency and discrimination of the new and original approaches to scoring the CBCL. Some of the new subscales demonstrated better sensitivity, positive predictive power, and discriminant validity than the original CBCL subscales; however, subscales from both approaches demonstrated low sensitivity. Results support the use of the new subscales for specific research purposes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings.

Two recent reviews have attempted to summarize findings quantitatively regarding assessment center (AC) construct-related validity (i.e., Lance, Lambert, Gewin, Lievens, & Conway, 2004; Lievens & Conway, 2001). Unlike these previous studies, which reanalyzed individual multitrait-multimethod (MTMM) matrices from previously published research, the authors recoded and combined past matrices into a single MTMM matrix. This matrix, comprised of 6 dimensions each measured by 6 exercises, was then analyzed, providing a more generalizable set of results. Both dimensions and exercises were found to contribute substantially to AC ratings. Specific dimensions (i.e., communication, influencing others, organizing and planning, and problem solving) appeared more construct valid than others (i.e., consideration/awareness of others and drive). Implications for AC design and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of assessment center ratings for the prediction of performance ratings and school climate of school administrators.

Describes the validation of an assessment center used to select school administrators on the basis of ratings of 153 school administrators who had participated in the center between 1976 and 1981. Behaviorally anchored rating scales were obtained from supervisors, teachers, and support staff; and measures of 7 school climate dimensions were provided by teachers, students, and support staff. Results show a significant relationship between an overall assessment center placement recommendation and supervisory, teacher, and support staff ratings on most performance dimensions. Climate measures were significantly related to few assessment-center ratings regardless of the source of the climate measure. It is suggested that the assessment center proved valid for a sample from widely distant geographic areas and school districts of differing sizes and levels. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Situational specificity of laboratory-induced learned helplessness.

Investigated the effects of exposure to escapable or inescapable noise in either the original pretreatment setting or in a dissimilar experimental setting with 80 undergraduates. In the same setting, Ss receiving inescapable noise displayed more anagram debilitation than did Ss receiving escapable noise. However, when inescapable Ss were removed from the original noise-exposure setting under the guise of participating in a different experiment, equivalent anagram performance impairments were not found. In addition, there were no differences between the escapable and inescapable groups in pre- and postnoise change in depression, hostility, or anxiety as measured by the Multiple Affect Adjective Check List. Results do not support the learned helplessness theory, which suggests that exposure to an uncontrollable event is psychologically debilitating across diverse situations. (24 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analysis of the multitrait-multimethod matrix by confirmatory factor analysis.

The multitrait–multimethod (MTMM) matrix permits examination of the convergent and discriminant validity of psychological measures. Estimation using confirmatory factor analysis (CFA), the predominant analytical technique, has often resulted in severe difficulties, such as out-of-range estimates and convergence problems. This article shows that an important special case of one of the more frequently advocated CFA models is not identified and is therefore not estimable. Because most MTMM data are likely to conform closely to this special case, resulting analyses suffer from empirical underidentification. Alternative CFA models are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Situation specificity of behavior and assessment center validation strategies: A rejoinder to Neidig and Neidig.

Responds to R. D. Neidig and P. J. Neidig's (see record 1984-13649-001) criticism of the present authors' (see record 1982-31460-001) conclusion that the failure to find consistency of assessment center dimension ratings across exercises constitutes a threat to the use of content validity to show job relatedness. The present authors contend that situational differences support the argument that the assessor judgment process is too complex to be justified on content-validity grounds alone. A reconceptualization of assessment centers is offered that relies on identifying critical managerial roles, designing exercises to simulate these roles, and evaluating effectiveness in each exercise. The conditions under which content validity is appropriate for showing the job relatedness of this type of center are specified. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A new, more powerful approach to multitrait-multimethod analyses: Application of second-order confirmatory factor analysis.

The advantages of applying confirmatory factor analysis (CFA) to multitrait–multimethod (MTMM) data are widely recognized. However, because CFA, as traditionally applied to MTMM data, incorporates single indicators of each scale (i.e., each trait–method combination), important weaknesses are the failure to (a) correct appropriately for measurement error in scale scores, (b) separate error due to low internal consistency from uniqueness due to weak trait or method effects, (c) test whether items or subscales accurately reflect the intended factor structure, and (d) test for correlated uniquenesses. However, when the analysis begins with multiple indicators of each scale (i.e., items or subscales), second-order factor analysis can be used to address each of these problems. In this approach, first-order factors defined by multiple items or subscales are posited for each scale, and the method and trait factors are posited as second-order factors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A confirmatory factor analysis of the structure of tacit knowledge in nursing

The purpose of this study was to examine the structure of tacit knowledge in the nursing profession. Using research in implicit learning and practical intelligence as a theoretical framework, a paper-and-pencil measure of tacit knowledge for nurses was developed and refined. Five models, each representing an established theory of traditional intelligence, were compared in terms of fit to the data. The results of the confirmatory factor analysis suggest that Model 3, a hierarchical model of intelligence, was the most plausible representation of tacit knowledge in nursing. The results indicate that the portion of managerial decision making learned implicitly on the job is mainly accounted for by managing tasks and others. Suggestions for nursing education include teaching effective strategies for managing tasks, such as handling increased workloads, establishing priorities, and delegating responsibility.     

Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory.

Current interest in the assessment of measurement equivalence emphasizes 2 major methods of analysis. The authors offer a comparison of a linear method (confirmatory factor analysis) and a nonlinear method (differential item and test functioning using item response theory) with an emphasis on their methodological similarities and differences. The 2 approaches test for the equality of true scores (or expected raw scores) across 2 populations when the latent (or factor) score is held constant. Both approaches can provide information about when measurrment nonequivalence exists and the extent to which it is a problem. An empirical example is used to illustrate the 2 approaches. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of a model of attention with confirmatory factor analysis.

Structural equation modeling (specifically, analysis of moment structures; J. L. Arbuckle, 1996) was used to evaluate the goodness of fit of a model of components of attention (A. E. Mirsky, B. J. Anthony, C. C. Duncan, M. B. Aheam, & S. G. Kellam, 1991) to neuropsychological test data from 2 samples. One sample consisted of psychiatrically normal persons with and without sleep-disordered breathing, and the other sample consisted of the adults studied by A. F. Mirsky et al. (1991), who gave rise to this model. That sample included psychiatric patients as well as normals. An exploratory data reduction procedure, principal-components analysis, suggested that attention might be conceptualized as composed of 4 independent elements or components: focus-execute, sustain, shift, and encode. Neither the proposed orthogonal model nor a model permitting correlated factors adequately fit either data set, suggesting that these 4 attention constructs are as yet not clearly validated in the measures used to assess them. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Importance ratings in job analysis: Note on the misinterpretation of factor analyses.

A common practice in job analysis involves having subject matter experts (SMEs) provide importance weights for the behaviors identified as characteristics of a given job, and then grouping those behaviors by factor analysis. Two problems with using factor analysis on these data are explored: (a) The factors that emerge from such an analysis are not interpretable as important dimensions of the job, and (b) job dimensions that SMEs agree are important will not emerge as factors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)