Incremental Validity in the Psychological Assessment of Children and Adolescents.

Incremental validity in the process of psychological assessment of children and adolescents is explored. The authors highlight the dependence of the incremental validity of assessment information on factors such as goal of assessment, other information available, base rate of the problem or outcome, age or gender of the child, and type of problem being assessed. The authors discuss the incremental validity of assessment information from alternate sources, methods, and constructs. In view of the limited number of studies directly relevant to incremental validity in child clinical assessments, the authors call for more clinically relevant research. To have the greatest impact on child and adolescent services, this research must be readily generalized and immediately relevant to actual clinical practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Introduction to the Special Section on Incremental Validity and Utility in Clinical Assessment.

This special section focuses on the incremental validity and utility of clinical assessment data. The lack of replicated incremental validity research limits the ability of psychologists to establish their assessment practices on a solid empirical footing. The articles in this section deal with conceptual, methodological, and content issues in the development and use of clinical measures. The authors addressed several aspects of incremental validity research, including (a) the evaluation of the magnitude of validity increments, (b) the use of incremental validity data for test development and validation, (c) the costs of assessment, (d) the use of multi-informant and multimethod assessments, and (e) the treatment utility of assessment. Obstacles limiting the use of incremental validity research to inform and guide clinical practice are also emphasized. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The impact of corrections for faking on the validity of noncognitive measures in selection settings.

In selection research and practice, there have been many attempts to correct scores on noncognitive measures for applicants who may have faked their responses somehow. A related approach with more impact would be identifying and removing faking applicants from consideration for employment entirely, replacing them with high-scoring alternatives. The current study demonstrates that under typical conditions found in selection, even this latter approach has minimal impact on mean performance levels. Results indicate about .1 SD change in mean performance across a range of typical correlations between a faking measure and the criterion. Where trait scores were corrected only for suspected faking, and applicants not removed or replaced, the minimal impact the authors found on mean performance was reduced even further. By comparison, the impact of selection ratio and test validity is much larger across a range of realistic levels of selection ratios and validities. If selection researchers are interested only in maximizing predicted performance or validity, the use of faking measures to correct scores or remove applicants from further employment consideration will produce minimal effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relative Efficacy of Criminological, Clinical, and Personality Measures of Future Risk of Offending in Mentally Disordered Offenders: A Comparative Study of HCR-20, PCL:SV, and OGRS.

The authors compared the ability of 3 commonly used measures of risk of future offending in a sample of 315 mentally disordered offenders discharged from a medium-secure unit in the United Kingdom. The authors explored whether the same criminogenic factors that predict recidivism in the general population also predict recidivism in mentally disordered offenders. The actuarial measure, using mainly criminological variables, provided the best prediction of recidivism compared with measures based on personality or clinical information, which provided no incremental validity over the actuarial measure. The authors suggest that for maximum efficacy clinical risk should be rated at a time of active symptoms rather than at discharge when symptoms are minimal. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Incremental Validity of Psychological Testing and Assessment: Conceptual, Methodological, and Statistical Issues.

There has been insufficient effort in most areas of applied psychology to evaluate incremental validity. To further this kind of validity research, the authors examined applicable research designs, including those to assess the incremental validity of test instruments, of test-informed clinical inferences, and of newly developed measures. The authors also considered key statistical and measurement issues that can influence incremental validity findings, including the entry order of predictor variables, how to interpret the size of a validity increment, and possible artifactual effects in the criteria selected for incremental validity research. The authors concluded by suggesting steps for building a cumulative research base concerning incremental validity and by describing challenges associated with applying nomothetic research findings to individual clinical cases. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The clinical utility of the Rorschach: Unfulfilled promises and an uncertain future.

The empirical evidence on the Rorschach is reviewed using three definitions of clinical utility: (a) the nature of professional attitudes and extent of clinical usage, (b) the extent of evidence for reliability, validity, diagnostic efficiency, and incremental validity, and (c) the extent of evidence that Rorschach data improve clinical decision-making and/or treatment outcome. Surveys demonstrate that the Rorschach is extensively used; however, these data are insufficient to demonstrate clinical utility as they do not address the rational, scientific, and ethical requirements of professional standards for psychological measures. After reviewing conceptual issues in Rorschach research (especially those in the Comprehensive System) the authors conclude that there is little scientific evidence to support the clinical utility of the Rorschach. Given the absence of data evaluating how the Rorschach is used in routine practice and whether its use is consistent with the manner in which it is used in research, there is currently no scientific basis for justifying the use of Rorschach scales in psychological assessments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Clarifying problems and offering solutions for correlated error when assessing the validity of selected-subtest short forms.

The correlation between a short-form (SF) test and its full-scale (FS) counterpart is a mainstay in the evaluation of SF validity. However, in correcting for overlapping error variance in this measure, investigators have overattenuated the validity coefficient through an intuitive misapplication of P. Levy's (1967) formula. The authors of the present article clarify that such corrections should be based on subtest-level versus FS-level data. Additionally, the authors propose a simple, modified equation incorporating FS-level scores that provides liberal and conservative validity measures for comparison across estimation methods, and they demonstrate its use in both a normative (N = 2,450) and clinical psychiatric (N = 216) sample. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity.

This review of 122 research reports (184 independent samples, 14,900 subjects) found average r = .274 for prediction of behavioral, judgment, and physiological measures by Implicit Association Test (IAT) measures. Parallel explicit (i.e., self-report) measures, available in 156 of these samples (13,068 subjects), also predicted effectively (average r = .361), but with much greater variability of effect size. Predictive validity of self-report was impaired for socially sensitive topics, for which impression management may distort self-report responses. For 32 samples with criterion measures involving Black–White interracial behavior, predictive validity of IAT measures significantly exceeded that of self-report measures. Both IAT and self-report measures displayed incremental validity, with each measure predicting criterion variance beyond that predicted by the other. The more highly IAT and self-report measures were intercorrelated, the greater was the predictive validity of each. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Operational Validity of a Video-Based Situational Judgment Test for Medical College Admissions: Illustrating the Importance of Matching Predictor and Criterion Construct Domains.

This study is part of a trend of examining noncognitive predictors, for example, a situational judgment test (SJT), as supplements to cognitive predictors for making college admission decisions. The authors examined criterion data over multiple academic years and universities. The criterion domain was broadly conceptualized, including both cognitive and interpersonal domains. The sample consisted of 7,197 candidates of the Medical and Dental Studies Admission Exam in Belgium. Results confirmed the importance of cognitive predictors. A video-based SJT was differentially valid for predicting overall grade point average for different curricula. The SJT showed incremental validity over cognitively oriented measures for curricula that included interpersonal courses, but not for other curricula. The SJT became more valid through the years. This demonstrates the importance of carefully specifying predictor- criterion linkages and of differentiating both predictor and criterion constructs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reconsidering vocational interests for personnel selection: The validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions.

Although vocational interests have a long history in vocational psychology, they have received extremely limited attention within the recent personnel selection literature. We reconsider some widely held beliefs concerning the (low) validity of interests for predicting criteria important to selection researchers, and we review theory and empirical evidence that challenge such beliefs. We then describe the development and validation of an interests-based selection measure. Results of a large validation study (N = 418) reveal that interests predicted a diverse set of criteria—including measures of job knowledge, job performance, and continuance intentions—with corrected, cross-validated Rs that ranged from .25 to .46 across the criteria (mean R = .31). Interests also provided incremental validity beyond measures of general cognitive aptitude and facets of the Big Five personality dimensions in relation to each criterion. Furthermore, with a couple exceptions, the interest scales were associated with small to medium subgroup differences, which in most cases favored women and racial minorities. Taken as a whole, these results appear to call into question the prevailing thought that vocational interests have limited usefulness for selection. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The incremental validity of vocational self-efficacy: An examination of interest, self-efficacy, and occupation.

The authors examined the incremental validity, beyond vocational interest, of the General Confidence Themes of the Skills Confidence Inventory (N. E. Betz, F. H. Borgen, & L. W. Harmon, 1996a) as measures of vocational self-efficacy in identifying tenured and satisfied membership in 21 occupational groups for 1,105 employed women and men. The General Occupational Themes of the Strong Interest Inventory (L. W. Harmon, J. C. Hansen, F. H. Borgen, & A. L. Hammer, 1994) were used to measure interest. The results replicated T. J. G. Tracey's (1997) finding that self-efficacy and interest form similar structures. Results also demonstrated the explanatory power of self-efficacy and interest and the incremental validity of self-efficacy. The authors suggest that the Skills Confidence Inventory is (a) a valid measure of tenured and satisfied occupational membership, (b) a distinct measure from interest but similar in structure, and (c) a potentially useful career assessment measure. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessment of obsessive-compulsive symptom dimensions: Development and evaluation of the Dimensional Obsessive-Compulsive Scale.

Although several measures of obsessive-compulsive (OC) symptoms exist, most are limited in that they are not consistent with the most recent empirical findings on the nature and dimensional structure of obsessions and compulsions. In the present research, the authors developed and evaluated a measure called the Dimensional Obsessive-Compulsive Scale (DOCS) to address limitations of existing OC symptom measures. The DOCS is a 20-item measure that assesses the four dimensions of OC symptoms most reliably replicated in previous structural research. Factorial validity of the DOCS was supported by exploratory and confirmatory factor analyses of 3 samples, including individuals with OC disorder, those with other anxiety disorders, and nonclinical individuals. Scores on the DOCS displayed good performance on indices of reliability and validity, as well as sensitivity to treatment and diagnostic sensitivity, and hold promise as a measure of OC symptoms in clinical and research settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validation of a self-report measure of unrealistic relationship expectations.

Cognitive models of intimate partner aggression implicate maladaptive relationship beliefs as antecedents to aggression and targets for intervention. However, existing self-report measures of relationship beliefs have failed to differentiate aggressive and nonaggressive individuals, raising questions about their assessment of and role in understanding aggression. To address these concerns, the authors developed and tested a new measure of unrealistic relationship beliefs in a sample of 453 community couples. Structural validity, concurrent validity, discriminant validity, internal consistency, and temporal stability of the new measure were examined. The final scale demonstrated adequate internal consistency (αs=.83-.84), test-retest reliability (rs=.68-.74), and concurrent validity (small to moderate associations with predicted variables). Unrealistic relationship beliefs significantly differentiated aggressive and nonaggressive men, and this association remained significant after the authors controlled for other related variables. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychometric evaluation of the Multidimensional Assessment of Fatigue Scale for use with pregnant and postpartum women.

Although fatigue is a common experience for pregnant women and new mothers, few measures of fatigue have been validated for use with this population. To address this gap, the authors assessed psychometric properties of the Multidimensional Assessment of Fatigue (MAF) scale, which was used in 2 independent samples of pregnant women. Results indicated that the psychometric properties of the scale were very similar across samples and time points. The scale possesses a high level of internal consistency, has good convergent validity with measures of sleep quality and depression, and discriminates well from a measure of social support. Contrary to previous evaluations of the MAF, data strongly suggest that the scale represents a unidimensional construct best represented by a single factor. Results indicate that the MAF is a useful measure of fatigue among pregnant and postpartum women. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Clinical assessment applications of ambulatory biosensors.

Ambulatory biosensor assessment includes a diverse set of rapidly developing and increasingly technologically sophisticated strategies to acquire minimally disruptive measures of physiological and motor variables of persons in their natural environments. Numerous studies have measured cardiovascular variables, physical activity, and biochemicals such as cortisol in psychopathology and treatment research. The physiological concomitants of many behavior and medical disorders and the benefits of a multimethod assessment strategy provide strong rationales for clinical applications of ambulatory biosensor measurement. A number of psychometric dimensions of evaluation are important in clinical applications of biosensor measurement, including accuracy and validity, reliability and consistency, clinical utility, incremental validity and utility, sensitivity to change, generalizability, cost benefits, and the conditional nature of dimensions of biomeasure evaluation. The authors review ambulatory biosensor methods and make recommendations for use of the technology. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Incremental validity of the Ego Impairment Index: A re-examination of Dawes (1999).

This article examined the incremental validity of the Ego Impairment Index (EII), a Rorschach measure of cognition, perception, and reasoning. R. M. Dawes (1999) borrowed 2 previously published data sets to assess the incremental validity of the EII. Dawes determined that in order for the EII to be considered a valid measure, the overall EII score should "outperform" 2 of the variables that compose the index. Using this approach, Dawes reported that the EII had "deficient" (p?=?.059) incremental validity. In this study, the incremental validity of the EII was re-examined using 1 of the data sets used by R. M. Dawes (1999). Applying the same strategy as Dawes, the author tested the incremental validity of the EII in predicting positive symptoms of schizophrenia, a more appropriate and clinically relevant criterion measure of impairment. The EII significantly predicted positive symptoms (p?     

Hierarchical organization of personality and prediction of behavior.

Two studies evaluated personality trait measures and Big Five factor measures for their accuracy in predicting important behavior criteria. The results of both studies showed that the narrower traits and the broader factors, thought to define 2 levels of a hierarchy of personality variables, separately predicted most criterion variables. However, the incremental validity of the personality trait measures (the degree to which the traits increased the criterion prediction achieved by the factors) was generally much larger than the incremental validity of the Big Five factor measures. It was concluded that aggregating personality traits into their underlying personality factors could result in decreased predictive accuracy due to the loss of trait-specific but criterion-valid variance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Proactive Personality Scale and objective job performance among real estate agents.

A sample of 131 real estate agents was used to examine the criterion validity of the Proactive Personality Scale (T. S. Bateman & J. M. Crant, 1993). A job performance index was computed for each agent from archival records of the number of houses sold, number of listings obtained, and commission income over a 9-month period. Experience, social desirability, general mental ability, and 2 of the Big Five factors—Conscientiousness and Extraversion—were controlled for, and the Proactive Personality Scale explained an additional 8% of the variance in the objective measure of agents' job performance. These results provide additional evidence for the criterion validity of the Proactive Personality Scale and suggest that specific personality measures can have incremental validity over the Big Five factors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Emotion regulation among school-age children: The development and validation of a new criterion Q-sort scale.

To foster the study of emotion regulation beyond infancy and toddlerhood, a new criterion Q-sort was constructed. In Study 1, Q-scales for emotion regulation and autonomy were developed, and analyses supported their discriminant validity. Study 2 further explored the construct validity of the Emotion Regulation Q-Scale within a sample of 143 maltreated and 80 impoverished children, aged 6 to 12 years. A multitrait-multimethod matrix and confirmatory factor analyses indicated impressive convergence among the Emotion Regulation Q-Scale and established measures of affect regulation. This new scale also was discriminable from measures of related constructs, including Q-sort assessments of ego resiliency. The use of this new measure was further supported by its ability to distinguish between maltreated and comparison children and between groups of well-regulated versus dysregulated children. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Team Role Test: Development and validation of a team role knowledge situational judgment test.

The main objectives in this research were to introduce the concept of team role knowledge and to investigate its potential usefulness for team member selection. In Study 1, the authors developed a situational judgment test, called the Team Role Test, to measure knowledge of 10 roles relevant to the team context. The criterion-related validity of this measure was examined in 2 additional studies. In a sample of academic project teams (N = 93), team role knowledge predicted team member role performance (r = .34). Role knowledge also provided incremental validity beyond mental ability and the Big Five personality factors in the prediction of role performance. The results of Study 2 revealed that the predictive validity of role knowledge generalizes to team members in a work setting (N = 82, r = .30). The implications of the results for selection in team environments are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)