Clinical Judgment in Science: Reply.

Replies to comments published by M. S. Schulz and R. J. Waldinger (see record 2005-11115-010), J. M. Wood and M. T. Nezworski (see record 2005-11115-011), and H. N. Garb and W. M. Grove (see record 2005-11115-012) on the original article by D. Westen and J. Weinberger (see record 2004-19091-002). Schulz and Waldinger (2005) make the important point that just as researchers can capitalize on the knowledge of experienced clinical observers through aggregation, they can aggregate the judgment of lay observers in assessing phenomena such as emotion. The reason, as they articulate, is that skills such as "reading" emotion from facial expression, tone of voice, posture, and the constellation of cues provided in everyday life are an area of expertise for most people, one that is now often called social or emotional intelligence. As psychometricians have known for years, one can increase reliability in many different ways. The comments by Wood and Nezworski (2005) and Garb and Grove (2005) do not address our central thesis--namely, the importance of distinguishing two meanings of clinical. The point of the sentence around which Wood and Nezworski (2005) build their comment was simply that the same biases widely attributed to clinicians are common in scientists as well--a point for which we would be delighted to take credit, but it is one that was actually made much more elegantly by the historian and philosopher of science Thomas Kuhn (1962). The authors respond to Wood and Nezworski's (2005) specific concerns about misrepresentation. In their comment, Garb and Grove (2005) challenge us to document our view that anticlinician prejudice is widespread among many academic clinical psychologists. As research on implicit prejudice suggests, surveys of academic clinical psychologists might indicate little about their implicit attitudes, as evident in Garb and Grove's apparent lack of recognition of the offensive nature of comparing a clinician's attempt to revise his or her understanding when the patient says "I don't think what you just said is right" to astrology and Barnum effects. We appreciate Garb and Grove's (2005) point about potential differential effects of training and experience on reliability and validity of clinical judgment. The data they cite are important and bear consideration. We would offer two caveats, however. Finally, we cannot help but note that this series of comments and our reply to them provide a prototypical example of "clinical" judgment in science--that is, subjective, informal aggregation of data, often leading to a "gestalt" judgment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Value of Pooling "Naive" Expertise: Comment.

Comments on the article by D. Westen and J. Weinberger (see record 2004-19091-002), which explored the benefits and limitations of clinical observation and judgment. Westen and Weinberger identify two categories of informants--clinicians and participants--but these categories could be expanded to include other observers who might have particular expertise or experience related to the phenomenon of interest. The type of expert best suited to provide observations depends on the type of expertise required. There are some domains, however, in which those with the greatest expertise are neither specially trained observers nor self-reporters but, rather, lay observers who have a native or learned ability to detect complicated social or psychological phenomena and make subtle discriminations. This type of expertise is often thought of as intuitive because it uses implicit knowledge that is not always accessible to conscious awareness or capable of being fully articulated. One way to harness this intuitive expertise effectively is to pool the judgments of multiple lay observers. Our research has led us to believe that lay observers' intuitive judgments about emotions may in fact capture important information that is lost when coders depend on more commonly used manualized approaches such as the Specific Affect Coding System (SPAFF; Gottman, McCoy, Coan, & Collier, 1996) and the Facial Action Coding System (Ekman & Friesen, 1978). In our zealousness to reduce variability among coders and to make our methods more exportable to other investigators, researchers risk losing the intuitive expertise that people naturally develop in making judgments about the world. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Jousting With Straw Men: Comment on Westen, Novotny, and Thompson-Brenner (2004).

Empirically supported treatments (ESTs) do not cure every patient, and the randomized trial is not a flawless methodology. Upon these often-noted and widely accepted points, D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004a; see record 2004-15935-005) built a critique of ESTs and EST research. However, important work developing effective, clinically relevant treatments for serious problems was omitted from the Westen et al. (2004a) review. Little documentation was offered for the purported "assumptions" of EST methodology that Westen et al. (2004a) criticized; and different review standards were applied to studies supporting versus those disagreeing with Westen et al.'s (2004a) views. Finally, the correlational research designs proposed as a remedy by Westen et al. (2004a) have far more serious weaknesses than randomized trials, thoughtfully applied to real-world clinical care. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Empirically Supported Psychotherapies: Comment on Westen, Novotny, and Thompson-Brenner (2004).

D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004; see record 2004-15935-005) suggested that efforts to identify empirically supported treatments are misguided because they are based on assumptions that are not appropriate for some types of treatment and patients. The authors of this comment argue that Westen and colleagues are simply incorrect when they assert that empirically supported treatments require that psychopathology must be highly malleable, that treatments must be brief, or that the samples studied are unrepresentative of the kinds of patients typically encountered in clinical practice--comorbidity is common in many clinical trials. Randomized controlled trials remain the most powerful way to test notions of causal agency. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Review of Clinical versus statistical prediction: A theoretical analysis and review of the evidence.

Reviews the book "Clinical versus statistical prediction: A theoretical analysis and review of the evidence" by Paul E. Meehl (see record 1996-97896-000). This book talks about a continuing debate among psychologists regarding the relative accuracy and efficiency of statistical (actuarial) predictions and those made by clinicians on the basis of subjective "understanding" of individual cases. This book represents the author's first published statement of his position. In the reviewers opinion, the author has succeeded admirably. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multiculturalism: Questions, Not Answers.

A reply by Richard B. Stuart regarding the comment by Stolle, Hutz, and Sommers-Flanagan (see record 2005-13212-019). The American Psychological Association (APA) guidelines on multiculturalism (APA, 2002b) exhort psychologists to recognize and control their personal biases and to strive to "apply culturally appropriate skills in clinical and other applied psychological practices" (p. 3). My article (see record 2004-10365-001) offered 12 suggestions aimed at helping psychologists follow these mandates. It was motivated by the fear that some well-intended attempts to appreciate diversity could inadvertently contribute to the very insensitivity they were intended to prevent. It is my view that group influences are extremely important in shaping individual identity, but the effect of culture on a given individual cannot be verified without consulting that person directly. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Paradox or contradiction?

Notes that in the April, 1955 issue of the American Psychologist, two young psychologists note the discrepancy between theory and practice in the clinical field, particularly during training, and a veteran says that he itches too but offers no suggestions for relief. It seems clear that our society can use many clinicians, a few experimentalists, and fewer who are half-and-half. On the other hand, scholarly tradition requires publication and the age requires it to seem scientific. Training is institutional for ends that are individual. Such observations could be compounded and argue for much franker facing of the sociology and economics of clinical psychology than has been given them. The author concludes that no one should be certified for the use of a clinical technique unless and until he can prove competence through results in the judgment of a group of his peers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Clinical judgmental biases: The moderating roles of counselor cognitive complexity and counselor client preferences.

Found that counselor individual differences in cognitive complexity (CC), but not preferences for client problems, moderate the cognitive processes that lead to bias in clinical judgment. A particularly robust and unambiguous clinical bias, known as diagnostic overshadowing (S. Reiss et al; see record 1982-25973-001), was selected for study. Consistent with behavior decision (R. Nisbett and L. Ross, 1980) and complexity theories (J. Bieri et al, 1966), counseling psychologists with lower CC were more likely to form biased clinical judgments than were counseling psychologists with higher CC (N?=?119). Implications of the significant and nonsignificant results for research in clinical judgment and for counselor education and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comment on "Research on human judgment and its application to clinical practice."

Contends that D. Faust (see record 1987-05373-001), in discussing the accuracy of judgments made by clinical psychologists, overlooked much of the research done on overall level of validity, incremental validity, the appropriateness of confidence ratings, the relation between experience and clinical judgment, and the reliability of diagnoses. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The psychologist as an instrument of prediction.

This selection system is meant to be a synthesis of a statistical and a clinical approach. Psychologists administered group and individual tests to applicants for copilot courses in the Scandinavian Airlines System. "During the years 1951-56 a total of 780 applicants were examined." Remaining and dismissed pilots were compared and the resulting data served as the criteria of the validity of the selection system. The psychologists discussed their evaluations until they agreed on ratings. The results suggest that in this instance clinical prediction was better than statistical. "There is no magic in the fact that psychologists, when given this leading role, can be more effective as predictors than batteries of tests." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Practicing clinical psychologists’ provision of sexual health care services.

Practicing clinical psychologists are likely to work with sexual health concerns as part of their clinical practice because of high prevalence rates and sexual problems as symptoms of mental or physical health problems and their pharmacological treatment. However, the majority of clinicians do not receive didactic or supervised clinical training. This survey of 188 practicing clinical psychologists in one Canadian city confirmed that, despite lack of training, many clinicians discussed sexual health concerns with their clients and used a variety of sex therapy techniques. This survey also revealed, however, that 60% of clinicians did not ask, or very infrequently asked, clients about sexual health. In general, lack of training affected level of comfort, and both may result in inadequate application of sex therapy techniques and treatment. The results of this survey indicate an ethical imperative to included sexuality training in current graduate curricula to adequately prepare psychologists to assess, refer, and treat sexual health concerns. The inclusion of sexuality-related topics in existing clinical graduate courses, an increase in sexuality-specific courses focused on assessment and intervention in graduate curricula, and broader options for continuing education for practicing clinical psychologists are recommended. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Causal client models in selecting effective interventions: A cognitive mapping study.

An important reason to choose an intervention to treat psychological problems of clients is the expectation that the intervention will be effective in alleviating the problems. The authors investigated whether clinicians base their ratings of the effectiveness of interventions on models that they construct representing the factors causing and maintaining a client's problems. Forty clinical child psychologists drew causal models and rank ordered interventions according to their expected effectiveness for 2 cases. The authors found that different clinicians constructed different causal models for the same client. Also, the authors found low to moderate agreement about the effectiveness of different interventions. Nevertheless, the authors could predict clinicians' ratings of effectiveness from their individual causal models. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Next Generation of Psychotherapy Research: Reply to Ablon and Marci (2004), Goldfried and Eubanks-Carter (2004), and Haaga (2004).

The commentaries on D. Westen, C. M. Novotny, and H. Thompson-Brenner's (2004; see records 2004-15935-005; 2004-15935-006; and 2004-15935-007) review suggest a number of questions, such as how an empirically informed clinician can integrate both applied and basic science into practice. The authors suggest recommendations for design and funding of psychotherapy research, including expanding the targets of intervention beyond categorical DSM-IV diagnoses, routinely comparing short- and longer- term variants of experimental treatments, revising funding mechanisms to facilitate the study of treatments and follow-up assessments of appropriate duration for the problems they are targeting, requiring at least one "dissenter" on every research team, discouraging exclusion criteria other than those a reasonable clinician would use in everyday practice, creating funding mechanisms for developing and testing treatments in the community, and including practicing clinicians among reviewers of grant proposals. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meehl's contribution to clinical versus statistical prediction.

Paul E. Meehl's work on the clinical versus statistical prediction controversy is reviewed. His contributions included the following: putting the controversy center stage in applied psychology; clarifying concepts underpinning the debate (especially his crucial distinction between ways of gathering data and ways of combining them) as well as establishing that the controversy was real and not concocted, analyzing clinical inference from both theoretical and probabilistic points of view, and reviewing studies that compared the accuracy of these 2 methods of data combination. Meehl's (1954/1996) conclusion that statistical prediction consistently outperforms clinical judgment has stood up extremely well for half a century. His conceptual analyses have not been significantly improved since he published them in the 1950s and 1960s. His work in this area contains several citation classics, which are part of the working knowledge of all competent applied psychologists today. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Clinical decision-making: How prepared are we?

The importance of decision-making skills for clinical psychologists has been established. This study investigates the amount of academic instruction received by graduate students in this subject area. Courses from American Psychological Association accredited clinical psychology programs were examined, along with surveys completed by academic training directors (DCTs). Findings indicate limited formal academic instruction within the area of clinical decision-making, despite DCT importance ratings. Coverage of decision-making topics rarely occurs in stand-alone courses. Fewer than 50% of programs cover decision theory or decision improvement strategies. Suggestions are offered to address the discrepancy between essential clinical judgment capacities and amount of provided instruction. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Research etiquette in the study of clinician's behavior.

A critique of research strategy and tactics in the investigation of the processes and functions of clinical psychologists is presented using the report of Sechrest, Gallimore, and Hersch (see record 2005-10655-001) as an example. It is argued that the continuation of studies using college students as an analogue to clinicians contributes little to the understanding of such processes or functions and that the methods of differential psychology are more appropriate than those of experimental psychology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An evaluation of criteria used by clinicians to infer pathology from figure drawings.

6 psychologists and 8 nonpsychologists were given a mixture of figure drawings made by 30 normal and 30 psychiatric adolescents and instructed to infer which drawings were by patients and to state the criteria used. All criteria were cross-validated on a new sample (N = 86). Valid criteria of pathology were "bizarre," "distorted," "incomplete," and "transparent." Valid criteria of normality were: "happy expression" and "nothing pathological." Many criteria used by clinicians led to inaccurate inferences. Nonpsychologists discriminated as well as psychologists (65% and 64% accuracy, respectively). An empirically based prediction formula discriminated with 79% accuracy on cross-validation. It is concluded that elimination of invalid criteria can lead to a sharpening of clinical judgment and more accuracy in evaluation of figure drawings. (59 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)