A validity study of functionally-based behavioral consultation with students with emotional/behavioral disabilities.

Eighteen students (K-11th grade) with emotional/behavioral disorders who were at-risk for change of placement to more restrictive settings participated. Construct validity of functional behavioral assessment (FBA) was assessed with a multifunction-multimethod matrix that showed excellent convergent and divergent agreement with combined FBA methods and functional analysis results. Treatment validity was assessed with random assignment to either a FBA with consultation condition (FBC) or to a behavioral consultation without FBA condition (BC). Growth curve analysis showed a significant decrease in inappropriate behavior from baseline to treatment for both treatment groups compared to a control group. The efficiency of teachers' implementation effected treatment validity. Effect sizes showed that regardless of treatment condition that interventions with good fidelity realized the largest effect (d = -1.14). Social and habilitative validity was assessed with telephone interviews conducted a year after treatment. Sixty-five percent attributed the students' problem behaviors to their thoughts or feelings, whereas only 25% attributed it to classroom variables. Many (57%) indicated the most helpful intervention would be individual counseling, whereas only 14% indicated a more involved classroom management plan would be useful. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Screening for emotional and behavioral risk among students with limited English proficiency.

Students with limited English proficiency (LEP) make up one of the fastest growing segments of the student population; however, LEP status is often related to poor academic and behavioral outcomes. Teacher-reported behavioral rating scales can be informative measurements to screen and identify students at risk for behavioral and emotional problems. Given the focus on early identification and the changing demographics in schools, it is important to research the validity of screening instruments, as well as more comprehensive assessments, among LEP students. Items on a teacher-rated screener and a comprehensive behavioral rating scale were evaluated for invariance across two language groups. Results showed that the majority of items could be considered invariant across groups. This study also examined how behavioral ratings on invariant measures differed at the mean level between LEP and non-LEP students. LEP students were rated as having more learning problems and fewer adaptive skills; no significant differences were found for internalizing and externalizing problems. Results highlight the need for further research on this topic. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Discriminant validity of the Behavior Rating Inventory of Executive Function Parent Form for children with attention-deficit/hyperactivity disorder.

Attention-deficit/hyperactivity disorder (ADHD) is a neurodevelopmental disorder, with concomitant executive function deficits often being the focus of empirical and clinical investigation. This study explored the validity of the Behavior Rating Inventory of Executive Function Parent Form (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) for children with ADHD and a matched control sample. Fifty-eight children with ADHD (69% with comorbidity) and 58 matched controls were found to exhibit statistically (p     

Assessing the social validity of behavioral interventions: A review of treatment acceptability measures.

Increased attention to the construct of social validity in school psychology research and practice has resulted in a focus on treatment acceptability, the extent to which interventions are considered appropriate, effective, and fair. Questionnaires represent the typical measurement approach to treatment acceptability. This article reviews treatment acceptability measures and practices that have been developed and used to assess the social validity of behavioral interventions. A comparative framework was developed and applied to provide a systematic critique of 9 treatment acceptability measures along such dimensions as content and purpose of the instrument, psychometric properties, scoring procedures and interpretation, and use of the measure in research in practice. No one instrument was selected to be the most comprehensive and it is argued that treatment acceptability assessment practices need to move beyond the traditional questionnaire format. Alternatives to rating scales are presented and directions for future research regarding the measurement of treatment acceptability are discussed. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Detection of youth at high risk for substance use disorders: A longitudinal study.

This study extends prior research (D. Clark, J. Cornelius, L. Kirisci, & R. Tarter, 2005) by determining whether variation in the developmental trajectories of liability to substance use disorder (SUD) is contributed by neurobehavioral disinhibition, parental substance use involvement, and demographic variables. The sample, participants in a long-term prospective investigation, consisted of 351 boys, evaluated at ages 10-12, 12-14, 16, 19, and 22, whose parents either had SUD or no adult psychiatric disorder. Neurobehavioral disinhibition in childhood, in conjunction with parental lifetime substance use/SUD, place the child at very high risk for SUD by age 22 if psychosocial maladjustment progresses in severity in early adolescence. These results indicate that monitoring social adjustment during the transition from childhood to mid-adolescence is important for identifying youth at very high risk for succumbing to SUD by young adulthood. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Examining the validity of self-report assessments of family functioning: A question of the level of analysis.

Family assessment instruments attempt to measure family functioning at a particular level of the family system: individual, dyad, or family as a whole. This article introduces the concept of level validity, that is, whether an assessment measures family functioning at the level that it was intended to measure. The authors argue that whenever higher-order factors (e.g., dyadic subsystems) are the target of a measure, these factors should explain variance that is independent of their lower-order constituents (e.g., individual-level characteristics). Previously published data targeting dyadic subsystems within the family were reanalyzed using a model that controls for lower-order effects. Dyad-level factors rarely emerged independent of individual-level factors and, when they did, they did not replicate across samples. The results suggest that level validity should be tested and reported along with other aspects of construct validity before accepting such measures as valid assessments of family functioning. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A field study of frame-of-reference effects on personality test validity.

As part of a test validation study at a major U.S.-based airline, the authors tested the effects of providing an "at work" frame-of-reference on the validity of the NEO Five-Factor Inventory among a sample of customer service supervisors (N = 206). Frame-of-reference moderated the validity of the Extraversion and Openness to Experience subscales after controlling for cognitive ability. In addition, the frame-of-reference personality test showed incremental validity over cognitive ability (ΔR2 = .16), but the standard personality test did not (ΔR2 = .05). The authors' discussion focuses on implications for personality theory and research and on implications for increasing the validity of personality tests in organizational settings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Children's internal representations, empathy and fantasy play: A validity study of the SCORS-Q.

The authors investigated relationships among internal representations, empathy, and affective and cognitive processes in fantasy play to test the validity of the Social Cognition and Object Relations Scale Q-Sort (SCORS--Q; D. Westen, 1995) with children. Eighty-six 8--10-year-olds were administered 8 Thematic Apperception Test cards, a standardized play task, and a self-report empathy measure. Teachers rated children's empathy and helpfulness. As predicted, internal representations were related to empathy, helpfulness, and quality of fantasy play. Developmental differences on the SCORS--Q were consistent with object relations theory and with results from the original SCORS. The findings support the value of internal representations as a means of understanding children's interpersonal functioning and contribute to the validity of the SCORS--Q for use with children. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Further validation of the IDAS: Evidence of convergent, discriminant, criterion, and incremental validity.

The authors explicated the validity of the Inventory of Depression and Anxiety Symptoms (IDAS; D. Watson et al., 2007) in 2 samples (306 college students and 605 psychiatric patients). The IDAS scales showed strong convergent validity in relation to parallel interview-based scores on the Clinician Rating version of the IDAS; the mean convergent correlations were .51 and .62 in the student and patient samples, respectively. With the exception of the Well-Being Scale, the scales also consistently demonstrated significant discriminant validity. Furthermore, the scales displayed substantial criterion validity in relation to Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) mood and anxiety disorder diagnoses in the patient sample. The authors identified particularly clear and strong associations between (a) major depression and the IDAS General Depression, Dysphoria and Well-Being scales, (b) panic disorder and IDAS Panic, (c) posttraumatic stress disorder and IDAS Traumatic Intrusions, and (d) social phobia and IDAS Social Anxiety. Finally, in logistic regression analyses, the IDAS scales showed significant incremental validity in predicting several DSM-IV diagnoses when compared against the Beck Depression Inventory-II (A. T. Beck, R. A. Steer, & G. K. Brown, 1996) and the Beck Anxiety Inventory (A. T. Beck & R. A. Steer, 1990). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating the validity of computerized content analysis programs for identification of emotional expression in cancer narratives.

Psychological interventions provide linguistic data that are particularly useful for testing mechanisms of action and improving intervention methodologies. For this study, emotional expression in an Internet-based intervention for women with breast cancer (n = 63) was analyzed via rater coding and 2 computerized coding methods (Linguistic Inquiry and Word Count [LIWC] and Psychiatric Content Analysis and Diagnosis [PCAD]). Although the computerized coding methods captured most of the emotion identified by raters (LIWC sensitivity = .88; PCAD sensitivity = .83), both over-identified emotional expression (LIWC positive predictive value = .31; PCAD positive predictive value = .19). Correlational analyses suggested better convergent and discriminant validity for LIWC. The results highlight previously unrecognized deficiencies in commonly used computerized content-analysis programs and suggest potential modifications to both programs that could improve overall accuracy of automated identification of emotional expression. Although the authors recognize these limitations, they conclude that LIWC is superior to PCAD for rapid identification of emotional expression in text. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Treatment gain for sexual offenders against children predicts reduced recidivism: A comparative validity study.

Objective: To determine whether pro-social treatment change in sexual offenders would predict reductions in recidivism beyond static and dynamic risk factors measured at pretreatment and whether different methods for assessing change based on self-reports and structured clinical rating systems would show convergent validity. Method: We compared 3 methods for assessing treatment change with a sample of adult male sexual offenders against children (n = 218) who completed a prison-based cognitive–behavioral treatment program between 1993 and 2000. The methods were measures of change derived from offender self-reports on a psychometric battery administered both pre- and posttreatment, change across treatment on the Violence Risk Scale: Sexual Offender Version (VRS:SO; Olver, Wong, Nicholaichuk, & Gordon, 2007), and posttreatment ratings on the Standard Goal Attainment Scaling for Sex Offenders (SGAS; Hogue, 1994). Offenders were followed up for an average of 12.24 years after release. Results: All measures of treatment gain were positively correlated, and all significantly predicted reductions in sexual recidivism, with values for the area under the receiver-operating characteristic curve ranging from .66 (SGAS) to .70 (VRS:SO). Survival analyses showed that measures of change based on the psychometric battery significantly predicted recidivism after controlling for both static and dynamic factors measured at pretreatment, while results for the VRS:SO were similar but failed to reach significance. Conclusions: Measures of treatment change based on offender self-reports and structured clinical rating systems show convergent and predictive validity, which suggests that effective treatment that targets dynamic risk factors leads to a reduction in sexual recidivism. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Investigating the construct validity of the dissociative taxon: Stability analyses of normal and pathological dissociation.

Research has suggested the existence of a pathological dissociative taxon. However, relatively little is known about this taxon. This study examined the 2-month retest stability of this taxon--together with other measures of dissociation and the Big Five--in a sample of 465 undergraduates. Contrary to expectation, taxon scores were only modestly stable and were substantially less stable than the other measures, including continuous indicators of dissociation. Furthermore, most individuals who were identified as taxon members at one assessment were classified as nonmembers at the other. These results challenge the existence of a pathological dissociative taxon. More generally, these data demonstrate that statistically identified taxa need to be explicated through the normal process of construct validation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Loss of emotional experience after traumatic brain injury: Findings with the startle probe procedure.

The authors used affective modulation of the eyeblink startle response to examine the impact of traumatic brain injury (TBI) on emotional reactions to pictures. Participants were 13 individuals with severe TBI and 24 controls. Participants were presented with pictures that differed in affective valence (e.g., mutilated bodies, erotic couples, and household objects) while the eyeblink startle response to an acoustic probe was measured. Startle amplitude was used to assess valence of emotional response, and startle latency was used to index interest in the pictures. Subjective ratings of the affect and arousal elicited by the various pictures were also obtained. TBI impaired startle potentiation to unpleasant pictures but not startle attenuation to pleasant pictures. Further, subjective ratings indicated that TBI participants found unpleasant pictures less arousing than did controls. The results are consistent with recent evidence of differential impairment in negative versus positive emotions after TBI and are discussed in relation to 2 competing explanations of startle modulation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A validation study of the Worthington Personal History Blank.

The validity of the Worthington PH blank was compared with the validity of standardized tests (Wonderlic Personnel Test, The Personnel Laboratory's Employment Test, Thurstone Temperament Schedule, and the "Dominance" and "Self-Sufficiency" scales of the Bernreuter PI) for a sample of 47 publishing company employees. The criterion was associates' ratings on traits common to those measured by the PH and the other tests. Neither the PH nor the test results were significantly related to the criterion ratings. However, the difference (in favor of the tests) between the contingency coefficients comparing the tests with the ratings and the coefficients comparing the PH with the ratings was significant at the .06 level of confidence. It was concluded that the objective tests were at least as good as the PH, and that use of the latter is not warranted in terms of cost. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A new frame for frame-of-reference training: Enhancing the construct validity of assessment centers.

The authors undertook a comprehensive examination of the construct validity of an assessment center in this study by (1) gathering many different types of evidence to evaluate the strength of the inference between predictor measures and constructs (e.g., reliability, accuracy, convergent and discriminant relationships), (2) introducing a theoretically relevant intervention (frame-of-reference [FOR] training) aimed at improving construct validity, and (3) examining the effect of this intervention on criterion-related validity (something heretofore unexamined in the assessment center literature). Results from 58 assessees and 122 assessors suggested that FOR training was effective at improving the reliability, accuracy, convergent and discriminant validity, and criterion-related validity of assessment center ratings. Findings are discussed in terms of implications and future directions for both FOR training and assessment center practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effectiveness of Early Screening for Externalizing Problems: Issues of Screening Accuracy and Utility.

Accurate, early screening is a prerequisite for indicated interventions intended to prevent development of externalizing disorders and delinquent behaviors. Using the Fast Track longitudinal sample of 396 children drawn from high-risk environments, the authors varied assumptions about base rates and examined effects of multiple-time-point and multiple-rater screening procedures. The authors also considered the practical import of various levels of screening accuracy in terms of true and false positive rates and their potential costs and benefits. Additional research is needed to determine true costs and benefits of early screening. However, the results indicate that 1st grade single- and multiple-rater screening models effectively predicted externalizing behavior and delinquent outcomes in 4th and 5th grades and that early screening is justified. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The validity and reliability of the Violence Risk Scale: A treatment-friendly violence risk assessment tool.

The development of risk assessment tools that use dynamic variables to predict recidivism and to inform and facilitate violence reduction interventions is the next major challenge in the field of risk assessment and management. This study is the first in a 2-step process to validate the Violence Risk Scale (VRS), a risk assessment tool that integrates violence assessment, prediction, and treatment. Ratings of the 6 static and 20 dynamic VRS variables assess the client's level of risk. Ratings of the dynamic variables identify treatment targets linked to violence, and ratings of the stages of change of the treatment targets assess the client's treatment readiness and change. The VRS scores of 918 male offenders showed good interrater reliability and internal consistency and could predict violent and nonviolent recidivism over both short- and longer term (4.4-year) follow-up. The probability of violent and nonviolent recidivism varied linearly with VRS scores. Dynamic and static variables performed equally well. The results support the contention that the VRS can be used to assess violent risk and to guide violence reduction treatment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Strong claims and weak evidence: Reassessing the predictive validity of the IAT.

The authors reanalyzed data from 2 influential studies—A. R. McConnell and J. M. Leibold (see record 2001-11532-008) and J. C. Ziegert and P. J. Hanges (see record 2005-05102-011)—that explore links between implicit bias and discriminatory behavior and that have been invoked to support strong claims about the predictive validity of the Implicit Association Test. In both of these studies, the inclusion of race Implicit Association Test scores in regression models reduced prediction errors by only tiny amounts, and Implicit Association Test scores did not permit prediction of individual-level behaviors. Furthermore, the results were not robust when the impact of rater reliability, statistical specifications, and/or outliers were taken into account, and reanalysis of A. R. McConnell & J. M. Leibold (2001) revealed a pattern of behavior consistent with a pro-Black behavioral bias, rather than the anti-Black bias suggested in the original study. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Child Attachment Interview: A psychometric study of reliability and discriminant validity.

While well-established attachment measures have been developed for infancy, early childhood, and adulthood, a "measurement gap" has been identified in middle childhood, where behavioral or representational measures are not yet sufficiently robust. This article documents the development of a new measure--the Child Attachment Interview (CAI)--which seeks to bridge this gap. The CAI is a semistructured interview, in which children are invited to describe their relationships with their primary caregivers. The coding system is informed by the Adult Attachment Interview and the Strange Situation Procedure, and produces 4 attachment categories along with a continuous measure of attachment security based on ratings of attachment-related dimensions. The main psychometric properties are presented, including interrater reliability, test-retest reliability, and concurrent and discriminant validities, both for normally developing children and for those referred for mental health treatment. The CAI correlates as expected with other attachment measures and predicts independently collected ratings of social functioning. The findings suggest that the CAI is a reliable, valid, and promising measure of child-parent attachment in middle childhood. Directions for improvements to the coding system are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Development and validation of a Coping with Discrimination Scale: Factor structure, reliability, and validity.

Four studies were conducted to develop and validate the Coping With Discrimination Scale (CDS). In Study 1, an exploratory factor analysis (N = 328) identified 5 factors: Education/Advocacy, Internalization, Drug and Alcohol Use, Resistance, and Detachment, with internal consistency reliability estimates ranging from .72 to .90. In Study 2, a confirmatory factor analysis (N = 328) provided cross-validation of the 5-factor model as well as evidence for validity of the scale. The validity evidence was similar across racial groups and for males and females. In Study 3, the estimated 2-week test–retest reliabilities (N = 53) were between .48 and .85 for the 5 factors. Education/Advocacy, Internalization, Drug and Alcohol Use, and Detachment were positively associated with active coping, self-blame, substance use, and behavioral disengagement, respectively, providing further support for validity of the CDS. Finally, incremental validity evidence was obtained in Study 4 (N = 220), where it was shown that the CDS explained variance in outcome variables (i.e., depression, life satisfaction, self-esteem, and ethnic identity) that could not be explained by general coping strategies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)