Rater (dis)agreement on risk assessment measures in sexually violent predator proceedings: Evidence of adversarial allegiance in forensic evaluation?

Actuarial risk assessment measures are often admitted in court, partly because strong psychometric properties such as interrater agreement suggest that they increase reliability and reduce subjectivity in forensic evaluation. But how strong is rater agreement when raters are retained by opposing sides in adversarial legal proceedings? The authors review sexual offender civil commitment cases in which opposing evaluators reported scores on the STATIC-99, the Minnesota Sex Offender Sex Offender Screening Tool—Revised (MnSOST–R), or the Psychopathy Checklist—Revised (PCL–R) for the same individual. Differences between scores from opposing evaluators were often greater than expected based on rater agreement values reported in the instrument manuals and research literature. Score differences were often in a direction that supported the party who retained each evaluator. Rater agreement was stronger for the STATIC-99, intraclass correlation coefficient ([ICC]A,1) = .64; than for the MnSOST–R, ICC(A,1) = .48; and the PCL–R, ICC(A,1) = .42. STATIC-99 scores appeared less influenced by adversarial allegiance. Overall, however, results raise concern that an evaluator's adversarial allegiance could influence some assessment instrument scores in forensic evaluation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Stability and discriminant validity of the Adult Attachment Interview: A psychometric study in young Israeli adults.

59 male and female Israeli students were interviewed twice by 2 different interviewers at 3 mo intervals to assess the Adult Attachment Interview's (AAI; C. George et al, 1985) test–retest reliability and the effects of the interviewers on the interview itself as well as its subsequent classification. Various memory measures were used to obtain a wide range of information about Ss' memory abilities. Information was also obtained from the students' records about various intelligence-related skills. Results showed high degree of interjudge and test–retest reliabilities, irrespective of interviewers. The classifications on the AAI were not found to be associated with nonattachment-related memory and intelligence abilities. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Beck Self-Concept Test.

The development and psychometric characteristics of the Beck Self-Concept Test (BST) are described. The BST is a 25-item self-report instrument that asks respondents to evaluate themselves in relation to other people whom they know. An item analysis with 550 psychiatric outpatients diagnosed primarily with DSM-III—R mood or anxiety disorders indicated that all of the BST's items were significantly correlated with the corrected total scores; the coefficient alpha was .82, indicating good internal consistency. The 1-week and 3-month test–retest reliabilities were .88 and .65, respectively. The BST was significantly related to the Rosenberg Self-Esteem Scale and displayed both discriminant and convergent validities with respect to measures of psychopathology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Is criminal behavior a central component of psychopathy? Conceptual directions for resolving the debate.

The development of the Psychopathy Checklist—Revised (PCL–R; R. D. Hare, 2003) has fueled intense clinical interest in the construct of psychopathy. Unfortunately, a side effect of this interest has been conceptual confusion and, in particular, the conflating of measures with constructs. Indeed, the field is in danger of equating the PCL–R with the theoretical construct of psychopathy. A key point in the debate is whether criminal behavior is a central component, or mere downstream correlate, of psychopathy. In this article, the authors present conceptual directions for resolving this debate. First, factor analysis of PCL–R items in a theoretical vacuum cannot reveal the essence of psychopathy. Second, a myth about the PCL–R and its relation to violence must be examined to avoid the view that psychopathy is merely a violent variant of antisocial personality disorder. Third, a formal, iterative process between theory development and empirical validation must be adopted. Fundamentally, constructs and measures must be recognized as separate entities, and neither reified. Applying such principles to the current state of the field, the authors believe the evidence favors viewing criminal behavior as a correlate, not a component, of psychopathy. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Assessment of psychopathy in a population of incarcerated adolescent offenders.

The reliability, validity, and factor structure of a modified version of the Psychopathy Checklist—Revised (PCL—R) for adolescents was investigated, completed using file information only, in a sample of incarcerated Black and White male adolescents. Interrater reliability and internal consistency were high, and confirmatory factor analyses and coefficients of congruence showed that the factor structure in this sample resembled the 2-factor solution found in adults. No significant racial differences were found for reliability or mean PCL—R scores. In addition, relationships between PCL—R scores and psychometric measures and behavioral indicators of maladjustment were similar to those previously found in adult populations. The construct of psychopathy, as defined by the PCL—R modified for use with adolescents, appears applicable to both Black and White adolescent male offenders. The study gives evidence for the structural and substantive validity of the modified PCL—R in this population. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Stability of Wechsler Adult Intelligence Scale—Revised factor scores across time.

This study was designed to assess the stability of Wechsler Adult Intelligence Scale—Revised (WAIS—R) factor scores across time. Information provided in the manual (D. Wechsler, 1981) on 2 subsets of the standardization data was used to estimate sample stability in Verbal Comprehension, Perceptual Organization, and Memory/Freedom from Distractibility scores. For both samples, and across all factors, estimated test–retest correlations were high. In addition, test–retest scores of 39 individuals with IQs     

Psychopathy, antisocial personality, and suicide risk.

H. Cleckley (1976) maintained that psychopaths are relatively immune to suicide, but substantial evidence exists for a relationship between antisocial deviance and suicidal acts. This study was the first to explicitly examine suicidal history among psychopathic individuals as defined by R. D. Hare's (1991) Psychopathy Checklist—Revised (PCL—R). Male prison inmates (N?=?313) were assessed using the PCL—R and DSM-III R and DSM-IV criteria (American Psychiatric Association, 1987, 1994) for antisocial personality disorder (APD), and they completed A. Tellegen's (1982) Multidimensional Personality Questionnaire (MPQ). Presence or absence of prior suicide attempts was coded from structured interview and prison file records. Suicide history was significantly related to PCL—R Factor 2 (which reflects chronic antisocial deviance) and to APD diagnosis but was unrelated to PCL-R Factor 1, which encompasses affective and interpersonal features of psychopathy. Higher order MPQ dimensions of Negative Emotionality and low Constraint were found to account for the relationship between history of suicidal attempts and antisocial deviance, indicating that temperament traits may represent a common vulnerability for both. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparability of computer- and booklet-administered Minnesota Multiphasic Personality Inventories among primarily chemically dependent patients.

Minnesota Multiphasic Personality Inventories were administered twice to chemically dependent and psychiatric patients to assess the equivalence of booklet- (B) and computer-administered (C) formats. 50 Ss each were assigned to BC, CB, BB, and CC conditions. The correlations between the formats were high and generally equal to their test–retest coefficients. The formats additionally demonstrated essentially equal scale variances, test–retest reliabilities, and invalid responding rates. In 3 of 4 analyses, computer-based scores underestimated their booklet counterparts slightly but not significantly. Computerized administration was faster and preferred by the Ss. The results support computerized administration, but our data and earlier studies (e.g., R. E. Lushene et al; see record 1975-00060-001) suggest that a slight change in norms may be needed to compensate for its apparent tendency to underestimate booklet-based scores. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Use of the Personality Assessment Inventory to assess psychopathy in offender populations.

The authors investigated the validity of the Antisocial Features (ANT) scale of the Personality Assessment Inventory (PAI; L. C. Morey, 1991) with respect to assessments of psychopathy in 2 offender samples. Study 1 included 46 forensic psychiatric inpatients who were administered the Screening Version of the Hare Psychopathy Checklist (PCL:SV; S. D. Hart, D. N. Cox, and R. D. Hare, 1995). In Study 2, 55 sex offenders were administered the Hare Psychopathy Checklist—Revised (PCL—R; R. D. Hare, 1991). ANT scores correlated highly with the PCL:SV total score (r?=?.54) and moderately with the PCL—R total score (r?=?.40). ANT tapped primarily behavioral symptoms of psychopathy rather than interpersonal and affective symptoms. Also, ANT had low to moderate diagnostic efficiency regarding diagnoses of psychopathy, suggesting that it may be better used as a dimensional rather than categorical measure of this construct. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Repeating items to estimate the test-retest reliability of task inventory ratings.

Little is known about the test–retest reliability of task inventories. Because they typically take hours to complete and require rating hundreds of job tasks, repeating an entire inventory after a short interval is unlikely to produce motivated raters; conversely, increasing the time interval between administrations may result in actual changes in job content that would underestimate the true reliability. By repeating only a portion of the inventory with a very small test–retest interval, rater motivation should not be seriously affected, and changes in job content can be ruled out as a determinant of the results. The repeated item approach was used on 207 respondents and 3 task inventories. For most of the rating scales studied, reliabilities were slightly lower (.70s to .80s) than those reported in a previous study (.90s). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Simple, but effective: Odd- and even-item short forms for the WAIS—R.

Two easy-to-administer Wechsler Adult Intelligence Scale—Revised (WAIS—R) short forms for test–retest situations were developed. One sample of 90 22–67 yr old male psychiatric inpatients and a cross-validation sample of 30 22–79 yr old psychiatric inpatients completed the full WAIS—R. Test results were scored in the standard fashion and for 2 short forms developed by an odd–even split on 9 of the 11 subtests. Verbal, Performance, and Full Scale IQs derived from both short form scales closely approximated standard-form WAIS—R IQs. Short-form subtest-scaled scores, however, were more discrepant from the standard-form subtest-scaled scores. The short forms' practical advantages are discussed. (8 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

One measure does not a construct make: Directions toward reinvigorating psychopathy research—reply to Hare and Neumann (2010).

In our article by J. L. Skeem & D. J. Cooke, (2010), we outlined the dangers inherent in conflating the Psychopathy Checklist—Revised (PCL–R; R. Hare, 1991) with psychopathy itself. In their response, R. Hare and C. Neumann (2010) seemed to agree with key points that the PCL–R should not be confused with psychopathy and that criminal behavior is not central to psychopathy; at the same time, they said we provided no clear directions for theory or research. In this rejoinder, we clarify our argument that progress in understanding the unobservable construct of psychopathy hinges upon setting aside procrustean dependence on a monofocal PCL–R lens to test (a) actual theories of psychopathy against articulated validation hierarchies and (b) the relation between psychopathy and crime. In specifying these conceptual and applied directions, we hope to promote constructive dialogue, further insights, and a new generation of research that better distinguishes between personality deviation and social deviance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Depressive personality: Reliability, validity, and relation to dysthymia.

The reliability and validity of K. Schneider's (1958) construct of depressive personality was evaluated in a sample of 177 outpatients, who were administered structured diagnostic and family history interviews, an extensive battery of inventories, and a 6-mo follow-up assessment. The criteria for depressive personality had moderate to good interrater reliability, internal consistency, and test–retest stability, and the assessment of depressive personality traits was not influenced by patients' clinical states. In addition, preliminary support for the convergent and discriminant validity of the depressive personality construct was obtained. Although there were significant relations between the depressive personality and diagnoses of dysthymia from the Diagnostic and Statistical Manual of Mental Disorders (DSM-III) and DSM-III—Revised (DSM-III—R), the depressive personality was not entirely subsumed by existing mood disorders. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Psychopathy and Offending From Adolescence to Adulthood: A 10-Year Follow-Up.

This study examined the predictive validity of the Hare Psychopathy Checklist: Youth Version (PCL:YV; A. E. Forth, D. S. Kosson, & R. D. Hare, 2003) from adolescence to early adulthood. The authors coded the PCL:YV using file information and collected criminal record information over a 10-year follow-up period on 157 boys, ages 12 through 18, referred to Youth Forensic Psychiatric Services for assessment in 1986. The risk for violence into early adulthood was greater among those with high PCL:YV scores than among those with low scores, even after controlling for conduct disorder, age at first offence, and history of violent and nonviolent offending. These results indicate that the PCL:YV provides meaningful information about young offenders' risk for violence into early adulthood. Clinical implications are discussed, with reference to pertinent ethical issues. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Criterion-related validity of the Psychopathic Personality Inventory in a prison sample.

The Psychopathic Personality Inventory (PPI; S. 0. Lilienfeld & B. P. Andrews, 1996), a self-report measure of psychopathic personality features, and R. D. Hare's (1991) Psychopathy Checklist–Revised (PCL–R) were administered to adult youthful offender prison inmates (N?=?50). As hypothesized, PPI scores were significantly correlated with scores on the PCL–R, providing evidence of concurrent validity for the PPI. Moreover, unlike existing self-report psychopathy measures, the PPI showed a moderate and positive correlation with PCL–R Factor 1 (i.e., the core personality traits of psychopathy). Discriminant function analysis using the optimal PPI total score value to predict PCL–R classifications of psychopath (n?=?10) and nonpsychopath (n?=?40) resulted in accurate classification of 86% of the cases (sensitivity?=?.50, specificity?=?.95). Results are discussed in terms of the relative merits of these 2 measures of psychopathy and the validation of the PPI for clinical use. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On WAIS—R difference scores in a psychiatric sample.

This report examines the psychometric integrity of the Wechsler Adult Intelligence Scale—Revised (WAIS—R) subscales, and the differences between them, in a sample of 229 psychiatric patients from 2 community mental health centers (ages 16 to 85). The results verify the overall alpha and split-half reliabilities of the instrument and indicate that greater caution needs to be exercised in clinically evaluating difference scores. Cutoff values presented in the manual appear too low to be of any statistical or diagnostic merit. Distributions for each of the 55 possible difference scores found in this sample are presented and provide a better guide for making nosological determinations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Visual acuity measurements by wall charts and ortho-rater tests.

Visual acuity scores obtained on Ortho-Rater plates were compared with visual acuity scores on duplicate wall charts, using letter and modified Landolt ring targets. Test and retest scores were obtained for 117 soldiers. (1) The two methods were of equal difficulty, except for slight discrepancies introduced by photographic reduction of the charts used in the Ortho-Rater. (2) The Ortho-Rater test-retest reliabilities were significantly higher than the wall chart reliabilities. (3) The correlations between the Ortho-Rater and wall chart scores were about as high as the reliabilities of the scores. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The reliability and validity of the Panic Disorder Self-Report: A new diagnostic screening measure of panic disorder.

This study examined the Panic Disorder Self-Report (PDSR), a new self-report diagnostic measure of panic disorder based on the 4th edition of the Diagnostic and Statistical Manual of Mental Disorders (American Psychiatric Association, 1994). PDSR diagnoses were compared with structured interview diagnoses of individuals with generalized anxiety disorder, social phobia, and panic disorder and nonanxious controls. Receiver operating characteristic analyses revealed that the PDSR showed 100% specificity and 89% sensitivity. The PDSR also demonstrated retest reliability, convergent and discriminant validity, and kappa agreement of .93 with a structured interview. Finally, the PDSR demonstrated clinical validity. Students who were identified as having panic disorder using the PDSR did not have significantly different scores on the Panic Disorder Severity Scale--Self-Report form (P. R. Houck, D. A. Speigel, M. K. Shear, & P. Rucci, 2002) than a panic disordered community sample. However, both groups had significantly higher scores than students identified as not meeting criteria for panic disorder. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Individual change after epilepsy surgery: Practice effects and base-rate information.

To evaluate the role of practice and to establish statistically meaningful methods for assessing cognitive outcome after epilepsy surgery, test–retest scores for 47 left (LTL) and 49 right (RTL) temporal lobectomy patients on the Wechsler Adult Intelligence Scale—Revised (WAIS—R) and Wechsler Memory Scale—Revised (WMS—R) were compared with the scores of 40 epilepsy patients who had not received lobectomies (SZCs). Reliable change indexes were calculated to control for measurement error, and base-rate tables for individual change were constructed for each variable before and after adjustment for observed practice effects. More frequent positive Full Scale IQ changes were noted among LTL than among RTL patients following surgery, whereas negative changes on the WMS-R General Memory and Verbal Memory Indexes were more common among the LTL patients than among either the RTL or SZC groups. When practice effects were controlled, the RTL patients also exceeded base-rate expectations for negative outcomes on the Verbal Memory Index. (PsycINFO Database Record (c) 2010 APA, all rights reserved)