Introduction to the Special Section on Incremental Validity and Utility in Clinical Assessment.

This special section focuses on the incremental validity and utility of clinical assessment data. The lack of replicated incremental validity research limits the ability of psychologists to establish their assessment practices on a solid empirical footing. The articles in this section deal with conceptual, methodological, and content issues in the development and use of clinical measures. The authors addressed several aspects of incremental validity research, including (a) the evaluation of the magnitude of validity increments, (b) the use of incremental validity data for test development and validation, (c) the costs of assessment, (d) the use of multi-informant and multimethod assessments, and (e) the treatment utility of assessment. Obstacles limiting the use of incremental validity research to inform and guide clinical practice are also emphasized. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Incremental Validity and the Assessment of Psychopathology in Adults.

Results from studies on incremental validity are described for (a) interviews, (b) personality inventories, (c) projective techniques, and (d) brief self-rated and clinician-rated measures. In some of the studies (clinical judgment studies), psychologists were given increasing amounts of information. In other studies (statistical prediction studies), increasing amounts of assessment information were entered into a statistical prediction rule. Although relatively little research has been conducted on incremental validity, results that have been obtained tend to favor the use of interviews, personality inventories, and brief self-rated measures. Results are generally less encouraging for projective techniques. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Incremental Validity in the Psychological Assessment of Children and Adolescents.

Incremental validity in the process of psychological assessment of children and adolescents is explored. The authors highlight the dependence of the incremental validity of assessment information on factors such as goal of assessment, other information available, base rate of the problem or outcome, age or gender of the child, and type of problem being assessed. The authors discuss the incremental validity of assessment information from alternate sources, methods, and constructs. In view of the limited number of studies directly relevant to incremental validity in child clinical assessments, the authors call for more clinically relevant research. To have the greatest impact on child and adolescent services, this research must be readily generalized and immediately relevant to actual clinical practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Incremental Validity Principles in Test Construction.

The authors articulate 5 basic principles for enhancing incremental validity, both among elements within a test and between tests, during test construction: (a) careful, precise articulation of each element or facet within the content domain; (b) reliable measurement of each facet through use of multiple, alternate-form items; (c) examination of incremental validity at the facet level rather than the broad construct level; (d) use of items that represent single facets rather than combinations of facets; and (e) empirical examination of whether there is a broad construct or a combination of separate constructs. Using these principles, the authors offer specific suggestions for modifications in 3 classic test construction approaches: (a) criterion keying, (b) inductive test construction, and (c) deductive test construction. Implementation of these suggestions is likely to provide theoretical clarification and improved prediction. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Incremental Validity of Psychological Testing and Assessment: Conceptual, Methodological, and Statistical Issues.

There has been insufficient effort in most areas of applied psychology to evaluate incremental validity. To further this kind of validity research, the authors examined applicable research designs, including those to assess the incremental validity of test instruments, of test-informed clinical inferences, and of newly developed measures. The authors also considered key statistical and measurement issues that can influence incremental validity findings, including the entry order of predictor variables, how to interpret the size of a validity increment, and possible artifactual effects in the criteria selected for incremental validity research. The authors concluded by suggesting steps for building a cumulative research base concerning incremental validity and by describing challenges associated with applying nomothetic research findings to individual clinical cases. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the Usefulness of Measures of Normal Personality for Clinical Assessment: Evidence of the Incremental Validity of the Revised NEO Personality Inventory.

As a means of examining the incremental validity of a normal personality measure in the prediction of selected Axis I and II diagnoses, 1,342 inpatient substance abusers completed the Revised NEO Personality Inventory (NEO-PI-R) and the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) and were assessed with structured clinical interviews to determine diagnostic status. Results demonstrated that scores from the NEO-PI-R (a) were substantially related to the majority of diagnoses, accounting for between 8% and 26% of the variance in the diagnostic criteria; (b) explained an additional 3% to 8% of the variability beyond 28 selected MMPI-2 scale scores; (c) increased diagnostic classification an additional 7% to 23% beyond MMPI-2 scale scores; and (d) were significantly more useful when examined at the facet trait level than at the domain trait level. Implications for incorporating measures of normal personality into clinical assessment batteries are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessment of Chronic Pain in Rehabilitation: Outcomes Measures in Clinical Trials and Clinical Practice.

Objective: Chronic pain, or pain that lasts for months or even years, is prevalent among patients in rehabilitation settings, and treatment is costly. People with chronic pain often suffer physical limitations and emotional distress. As new treatments become available and clinical trials are initiated, it is important to measure outcomes in an effort to determine the efficacy of treatment interventions. This article provides a review of the core domains of outcomes assessment among people with chronic pain, including recommendations for outcome measures to be used in clinical trials and clinical practice settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Indigenous Measures of Personality Assessment in Asian Countries: A Review.

This article reviews attempts to develop multidimensional personality measures in Asia and their applications in clinical assessment. Indigenous personality assessment measures in India, Korea, Japan, the Philippines, and Taiwan are examined. These early attempts have not yielded a comprehensive personality measure that integrates a theoretical framework and an empirical program of validation. The Chinese Personality Assessment Inventory (CPAI) is cited as an example to illustrate the process of developing an indigenous measure that meets the testing standards of established assessment instruments. On the basis of the research findings from the CPAI, the authors discuss the relevance of indigenous measures in clinical assessment in native cultures as well as in informing mainstream personality assessment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On Construct Validity: Issues of Method and Measurement.

Fifty years ago, L. J. Cronbach and P. E. Meehl (1955) advocated for the concept of construct validity, noting that psychologists study hypothetical, inferred entities and that validating measures of such entities involves basic theory testing. Three important developments in clinical assessment following that seminal article are noteworthy. First, clinical research has benefited from greater theoretical integration and subsequent differentiation among related constructs. Second, implementation of ongoing, critical evaluation of all aspects of the construct validity process, including theory development, hypothesis specification, research design, and empirical evaluation, has improved clinical assessment. Third, improvement in evaluating fit between hypotheses and observations has been sought. Improved means of evaluating multitrait, multimethod designs, and ways to increase their clinical representativeness, are one encouraging development. Ongoing efforts to improve the construct validity process reflect the legacy of L. J. Cronbach and P. E. Meehl. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Impact of Elaboration on Socially Desirable Responding and the Validity of Biodata Measures.

The current study investigated the impact of requiring respondents to elaborate on their answers to a biodata measure on mean scores, the validity of the biodata item composites, subgroup mean differences, and correlations with social desirability. Results of this study indicate that elaborated responses result in scores that are much lower than nonelaborated responses to the same items by an independent sample. Despite the lower mean score on elaborated items, it does not appear that elaboration affects the size of the correlation between social desirability and responses to biodata items or that it affects criterion-related validity or subgroup mean differences in a practically significant way. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Intelligence and Maturity: Meta-Analytic Evidence for the Incremental and Discriminant Validity of Loevinger's Measure of Ego Development.

This review examined whether Loevinger's measure of personality (ego) development is equivalent to the measurement of intelligence. The authors conducted a meta-analysis of 52 correlations between ego level scores and intelligence test scores (retrieved from 42 studies involving 5,648 participants). The weighted average correlation between ego level and intelligence ranged from .20 to .34, depending on the intellectual ability assessed (e.g., verbal intelligence). Adjusting for measurement unreliability increased these values only minimally. The authors also reviewed 16 studies that examined the association between ego level and various criterion variables (e.g., aggressive behavior) after statistically controlling for the effects of intelligence. Ninety-four percent of the tests revealed significant relations between ego level and criterion variables after controlling for intelligence, indicating that ego development and intelligence are not interchangeable constructs. These findings do not support recent speculations concerning the limited value of stage models of maturity, social development, and moral reasoning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the Complexity of Quantifying Construct Validity: Reply.

To validate a measure of a construct is to validate a theory: The process includes theory specification, theory elaboration, choice of methods, choice of research design, reliance on necessary auxiliary theories, and empirical observation. Evaluating the success of the process is likely to be enhanced through quantification. To date, scientists have quantified the final step of empirical observation, but earlier steps in the validation process appear more difficult to measure. In this issue, D. Westen and R. Rosenthal (2005; see record 2005-16347-003) emphasize that threats to validity are often methodological ones that precede empirical observation. Yet, their earlier attempt to quantify construct validity focused only on the final, empirical observation step of the process. For that reason, it can produce positive results despite real theoretical and methodological threats to validity. Nonetheless, the laudable attempt by these distinguished authors to quantify construct validity reminds us of the complexity of the construct validation process. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Concept of Validity.

This article advances a simple conception of test validity: A test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variation in the measurement outcomes. This conception is shown to diverge from current validity theory in several respects. In particular, the emphasis in the proposed conception is on ontology, reference, and causality, whereas current validity theory focuses on epistemology, meaning, and correlation. It is argued that the proposed conception is not only simpler but also theoretically superior to the position taken in the existing literature. Further, it has clear theoretical and practical implications for validation research. Most important, validation research must not be directed at the relation between the measured attribute and other attributes but at the processes that convey the effect of the measured attribute on the test scores. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Clinical validation of the Chinese Personality Assessment Inventory.

The clinical validity of the Chinese Personality Assessment Inventory (CPAI; F. M. Cheung, K. Leung, et al., 1996) was examined in 2 studies involving a group of 167 male prisoners in Hong Kong and a group of 339 psychiatric patients in China. Elevated scores on the clinical scales were obtained for the clinical samples. Logistic regression analyses confirmed that the CPAI scales were useful in differentiating between male prisoners and the Hong Kong male normative sample and between psychiatric patients and a random sample of normal adults in China. Multivariate analyses of variance results showed significant differences on the CPAI clinical scales and personality scales among subgroups of psychiatric patients with diagnoses of bipolar, schizophrenic, and neurotic disorders. The usefulness of an indigenous personality inventory is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessment of Substance Cue Reactivity: Advances in Reliability, Specificity, and Validity.

An arousal-control and cross-over design was used to evaluate the reliability, specificity, and validity of the Normative Appetitive Picture System (NAPS), a cue exposure protocol with sets of visual alcohol, cigarette, and control cues. The authors also examined the utility of conceptualizing cue reactivity as a multidimensional phenomenon involving independent approach and avoidance dimensions. University student participants (n=369) rated multiple cue images in terms of arousing properties and capacity to elicit separate approach and avoidance inclinations. They also completed a battery of substance-related individual-difference measures. Results indicated that NAPS protocol reactivity profiles had good reliability and high specificity across cue types and individuals with different substance use histories. Avoidance reactivity independently predicted self-reports of substance-related behaviors, after controlling for approach reactivity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Theory and utility-Key themes in evidence-based assessment: Comment on the special section.

This article focuses on two key themes in the four featured reviews on evidence-based assessment. The first theme is the essential role of theory in psychological assessment. An overview of this complex, multilayered role is presented. The second theme is the need for a common metric with which to gauge the utility of specific psychological tests and measures for specific purposes. A metric from information theory is recommended. The implications of these themes for the four reviews and for the future of psychological assessment in general are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Moderating effects and differential reliability and validity.

Classic psychometric theory holds that errors of measurement and of prediction are of the same magnitude for all individuals. Interactive effects are not recognized, and the psychological structure of all individuals is taken to be the same. To increase reliability and validity of measurement, then, attention is entirely focused on improvement of measuring devices. However, a substantial body of evidence indicates there are systematic individual differences in error, and in the importance a given trait has in determining a particular performance. Reliability and validity of measurement can be increased by the use of moderator variables which predict individual differences in error and in the importance of traits. (16 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Incremental Validity of the Frame-of-Reference Effect in Personality Scale Scores: A Replication and Extension.

Context-specific personality items provide respondents with a common frame of reference unlike more traditional, noncontextual personality items. The common frame of reference standardizes item interpretation and has been shown to reduce measurement error while increasing validity in comparison to noncontextual items (M. J. Schmit, A. M. Ryan. S. L. Stierwalt. & S. L. Powell, 1995). Although the frame-of-reference effect on personality scales scores has been well investigated (e.g., M. J. Schmit et al., 1995), the ability of this innovation to obtain incremental validity above and beyond the well-established, noncontextual personality scale scores has yet to be examined. The current study replicates and extends work by M. J. Schmit et al. (1995) to determine the incremental validity of the frame-of-reference effect. The results indicate that context-specific personality items do indeed obtain incremental validity above and beyond both noncontextual items and cognitive ability, and in spite of socially desirable responding induced by applicant instructions. The implications of these findings for personnel selection are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Use of Outcome Measures by Psychologists in Clinical Practice.

This survey investigated psychologists' use of outcome measures in clinical practice. Of the respondents, 37% indicated that they used some form of outcome assessment in practice. A wide variety of measures were used that were rated by the client or clinician. Clinicians who assess outcome in practice are more likely to be younger, have a cognitive-behavioral orientation, conduct more hours of therapy per week, provide services for children and adolescents, and work in institutional settings. Clinicians who do not use outcome measures endorse practical (e.g., cost, time) and philosophical (e.g., relevance) barriers to their use. Both users and nonusers of outcome measures were interested in similar types of information, including client progress since entering treatment, current strengths and weaknesses, and determining if there is a need to alter treatment. Implications for practicing clinicians are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development and preliminary validation of a scale of psychache.

Details the construction and psychometric evaluation of a scale of psychache. Two studies were conducted to describe the Psychache Scale's development and construct validity with 505 university undergraduates. Analyses indicate that the scale of psychache had strong internal consistency reliability, criterion validity for indicating various suicide criteria, and incremental validity, relative to other established psychological antecedents of suicide measures, for statistically predicting status on these suicide criteria. Findings support both the construct validity of the Psychache Scale and E. S. Shneidman's (1993) theory that emphasizes the pre-eminence of psychache as a cause of suicide. (PsycINFO Database Record (c) 2010 APA, all rights reserved)