Clinical Judgment in Science: Reply.

Replies to comments published by M. S. Schulz and R. J. Waldinger (see record 2005-11115-010), J. M. Wood and M. T. Nezworski (see record 2005-11115-011), and H. N. Garb and W. M. Grove (see record 2005-11115-012) on the original article by D. Westen and J. Weinberger (see record 2004-19091-002). Schulz and Waldinger (2005) make the important point that just as researchers can capitalize on the knowledge of experienced clinical observers through aggregation, they can aggregate the judgment of lay observers in assessing phenomena such as emotion. The reason, as they articulate, is that skills such as "reading" emotion from facial expression, tone of voice, posture, and the constellation of cues provided in everyday life are an area of expertise for most people, one that is now often called social or emotional intelligence. As psychometricians have known for years, one can increase reliability in many different ways. The comments by Wood and Nezworski (2005) and Garb and Grove (2005) do not address our central thesis--namely, the importance of distinguishing two meanings of clinical. The point of the sentence around which Wood and Nezworski (2005) build their comment was simply that the same biases widely attributed to clinicians are common in scientists as well--a point for which we would be delighted to take credit, but it is one that was actually made much more elegantly by the historian and philosopher of science Thomas Kuhn (1962). The authors respond to Wood and Nezworski's (2005) specific concerns about misrepresentation. In their comment, Garb and Grove (2005) challenge us to document our view that anticlinician prejudice is widespread among many academic clinical psychologists. As research on implicit prejudice suggests, surveys of academic clinical psychologists might indicate little about their implicit attitudes, as evident in Garb and Grove's apparent lack of recognition of the offensive nature of comparing a clinician's attempt to revise his or her understanding when the patient says "I don't think what you just said is right" to astrology and Barnum effects. We appreciate Garb and Grove's (2005) point about potential differential effects of training and experience on reliability and validity of clinical judgment. The data they cite are important and bear consideration. We would offer two caveats, however. Finally, we cannot help but note that this series of comments and our reply to them provide a prototypical example of "clinical" judgment in science--that is, subjective, informal aggregation of data, often leading to a "gestalt" judgment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the Merits of Clinical Judgment: Comment.

Comments on the article by D. Westen and J. Weinberger (see record 2004-19091-002), which criticized academic clinical psychologists for being cynical about clinical judgment and clinical practice. In our view, it seems unlikely that more than a few academic clinical psychologists believe that they have little to learn from clinical practice or experience. In this comment, we examine the arguments about clinical judgment made by Westen and Weinberger (2004). Westen and Weinberger (2004) conflate the effect of training with the effect of experience. Westen and Weinberger (2004) do not mention that the value of training in psychology has been well-supported by research. While Westen and Weinberger (2004) make positive comments about the types of feedback that clinicians receive, for a number of reasons, including the Barnum effect, psychologists can be misled by feedback. Westen and Weinberger (2004) also argue that "psychotherapists tend to have much more direct and immediate feedback than most other medical practitioners, who may prescribe a medication or perform a procedure and not see the patient again for a year" (p. 603). But when psychologists make a diagnosis or describe a personality trait, they frequently do not receive "direct and immediate feedback" on whether they are right or wrong. In contrast, physicians often receive highly valid feedback. Finally, in discussing the value of ratings made by clinicians, Westen and Weinberger (2004) observe that "empirically, we have found surprisingly little evidence of theory-driven observational bias in using clinician-report methods" (p. 601). The issue is important because Westen and Weinberger argue in favor of using clinician ratings to construct diagnostic criteria. If clinicians' ratings are biased, then the criteria will be biased. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Value of Pooling "Naive" Expertise: Comment.

Comments on the article by D. Westen and J. Weinberger (see record 2004-19091-002), which explored the benefits and limitations of clinical observation and judgment. Westen and Weinberger identify two categories of informants--clinicians and participants--but these categories could be expanded to include other observers who might have particular expertise or experience related to the phenomenon of interest. The type of expert best suited to provide observations depends on the type of expertise required. There are some domains, however, in which those with the greatest expertise are neither specially trained observers nor self-reporters but, rather, lay observers who have a native or learned ability to detect complicated social or psychological phenomena and make subtle discriminations. This type of expertise is often thought of as intuitive because it uses implicit knowledge that is not always accessible to conscious awareness or capable of being fully articulated. One way to harness this intuitive expertise effectively is to pool the judgments of multiple lay observers. Our research has led us to believe that lay observers' intuitive judgments about emotions may in fact capture important information that is lost when coders depend on more commonly used manualized approaches such as the Specific Affect Coding System (SPAFF; Gottman, McCoy, Coan, & Collier, 1996) and the Facial Action Coding System (Ekman & Friesen, 1978). In our zealousness to reduce variability among coders and to make our methods more exportable to other investigators, researchers risk losing the intuitive expertise that people naturally develop in making judgments about the world. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Amplifying issues related to psychological testing and assessment.

Responds to comments by D. A. Smith (see record 2002-10716-011), H. N. Garb et al (see record 2002-10716-012), R. Fernández-Ballesteros (see record 2002-10716-013), J. Hunsley (see record 2002-10716-014) regarding the G. J. Meyer et al (see record 2001-00159-003) summary of evidence and issues associated with psychological assessment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Jousting With Straw Men: Comment on Westen, Novotny, and Thompson-Brenner (2004).

Empirically supported treatments (ESTs) do not cure every patient, and the randomized trial is not a flawless methodology. Upon these often-noted and widely accepted points, D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004a; see record 2004-15935-005) built a critique of ESTs and EST research. However, important work developing effective, clinically relevant treatments for serious problems was omitted from the Westen et al. (2004a) review. Little documentation was offered for the purported "assumptions" of EST methodology that Westen et al. (2004a) criticized; and different review standards were applied to studies supporting versus those disagreeing with Westen et al.'s (2004a) views. Finally, the correlational research designs proposed as a remedy by Westen et al. (2004a) have far more serious weaknesses than randomized trials, thoughtfully applied to real-world clinical care. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analytic methods, the Rorschach, and the MMPI.

In response to concerns described by H. N. Garb et al(see record 2001-05665-003), the authors present the weighted and unweighted means and medians of the effect sizes obtained by J. B. Hiller et al (see record 1999-11130-005). These indices of central tendency are presented separately for MMPI and Rorschach effect sizes, both for all the studies in the meta-analysis and for a 10% trimmed sample designed to obtain more robust estimates of central tendency. The variability of these 4 indices is noticeably greater for the MMPI than for the Rorschach. Meta-analysts must compute, compare, and evaluate a variety of indices of central tendency, and they must examine the effects of moderator variables. The authors also comment briefly on the use of phi versus kappa, combining correlated effect sizes and possible hindsight biases. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Empirically Supported Psychotherapies: Comment on Westen, Novotny, and Thompson-Brenner (2004).

D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004; see record 2004-15935-005) suggested that efforts to identify empirically supported treatments are misguided because they are based on assumptions that are not appropriate for some types of treatment and patients. The authors of this comment argue that Westen and colleagues are simply incorrect when they assert that empirically supported treatments require that psychopathology must be highly malleable, that treatments must be brief, or that the samples studied are unrepresentative of the kinds of patients typically encountered in clinical practice--comorbidity is common in many clinical trials. Randomized controlled trials remain the most powerful way to test notions of causal agency. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Further comments toward a dimensional classification of personality disorder.

Responds to the comments by H. N. Garb (see record 2007-19520-012) and A. M. Ruscio (see record 2007-19520-013) on the current authors' original article "Plate tectonics in the classification of personality disorder: Shifting to a dimensional model" (see record 2007-01685-001). Unable to respond to all of Garb's and Ruscio's concerns given space limitations, the current authors attempt to respond to key points regarding their article on integrating the classification of personality disorder with a dimensional model of general personality structure. These points include: clinical judgments; feasibility; communication; thresholds; and validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Research on human judgment and its application to clinical practice": Response to Garb.

In response to the comment by H. N. Garb (see record 1988-18224-001) on the article by the present author (see record 1987-05373-001) concerning research on human judgment and its application, the present author refutes Garb's criticisms and maintains that many core aspects of the original argument remain untouched. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Deny nothing, doubt everything: A comment on Westen and Morrison (2001).

It is remarkable that so many important issues in psychotherapy research are touched on and, in some cases, more thoroughly explored in D. Westen and K. Morrison's (2001) meta-analysis (see record 2001-05666-001) and its discussion. Although no previously unanswered questions are resolved on the basis of their findings, original approaches to familiar questions are attempted and intriguing data are presented. Westen and Morrison's capacity to "think outside the box" while they seek to answer very familiar questions is most impressive. In the final analysis, the most significant outcome of Westen and Morrison's laudable effort can be that it might lead other psychotherapy researchers to do important things differently in the future. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the Need for a New Psychotherapy Research Paradigm: Comment on Westen, Novotny, and Thompson-Brenner (2004).

Basic, process, and outcome research have the potential to inform clinical practice. However, as D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004; see record 2004-15935-005) observed in their timely analysis, the current dominant paradigm for psychotherapy outcome research--the randomized clinical trial--is not fulfilling this potential. The field's reliance on the medical model and manual-based interventions has contributed to the gap between research and clinical practice. Greater collaboration between practitioners and researchers, a focus on therapeutic principles rather than treatment packages, and systemic changes in how scholarly efforts are reinforced are needed to shift the current research paradigm. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The status of empirically supported therapies: Comment on Westen and Morrison's (2001) multidimensional meta-analysis.

In this commentary, a meta-analytic review by D. Westen and K. Morrison (2001) (see record 2001-05666-001) is put in a historical context of the controversies over the effects of psychotherapy. It is suggested that recalculations of the effects of psychotherapy have been common in past debates and that Westen and Morrison's estimates understate the effects of therapy. Data are summarized that suggest that outcomes in clinical practice are equivalent to clinical trials research, but only when patients receive similar levels of treatment and regardless of whether empirically supported therapies are offered. Treatment gains are generally maintained. It is suggested that outcomes management strategies would be far more efficient than efficacy and effectiveness research in maximizing patient outcomes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

SPA is subliminal, but is it psychodynamically activating?

Responds to comments by J. Weinberger (see record 1990-04808-001) on Balay and Shevrin's (B&S; see record 1988-20203-001) work on the subliminal psychodynamic activation (SPA) method, which, contrary to Weinberger's assertion, was not an attempt to destroy the work of L. H. Silverman (published 1974–1985) and others who have used the SPA method. Rather, B&S hoped to encourage investigators using this method to take a closer look at the theoretical assumptions underlying SPA (i.e., the generic vs specific nature of subliminal stimuli that might activate conflict-related, unconscious fantasies). Analyses are also offered of comments made by T. E. Moore (see record 1990-04745-001), M. D. Figueroa (see record 1990-04666-001), and D. K. Silverman (see record 1990-04785-001). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Healthy Dose of Criticism for Randomized Trials: Comment on Westen, Novotny, and Thompson-Brenner (2004).

D. Westen, C. M. Novotny, and H. Thompson-Brenner (2004; see record 2004-15935-005) identified many important concerns in their critique of methods typically used in randomized controlled trials (RCTs) of psychotherapy outcome and by extension in methods of identifying empirically supported therapies (ESTs). Some of the concerns would be mitigated if empirical support of treatments were assessed multidimensionally (separating favorability of results from definitiveness of research methods used) and continuously rather than categorically. Other concerns can and should be addressed within the existing framework of RCTs and ESTs, including consideration of inclusion criteria other than a single Axis I condition, experimental evaluation of some of the procedural assumptions codified in psychotherapy manuals, and far more detailed reporting of RCT results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Next Generation of Psychotherapy Research: Reply to Ablon and Marci (2004), Goldfried and Eubanks-Carter (2004), and Haaga (2004).

The commentaries on D. Westen, C. M. Novotny, and H. Thompson-Brenner's (2004; see records 2004-15935-005; 2004-15935-006; and 2004-15935-007) review suggest a number of questions, such as how an empirically informed clinician can integrate both applied and basic science into practice. The authors suggest recommendations for design and funding of psychotherapy research, including expanding the targets of intervention beyond categorical DSM-IV diagnoses, routinely comparing short- and longer- term variants of experimental treatments, revising funding mechanisms to facilitate the study of treatments and follow-up assessments of appropriate duration for the problems they are targeting, requiring at least one "dissenter" on every research team, discouraging exclusion criteria other than those a reasonable clinician would use in everyday practice, creating funding mechanisms for developing and testing treatments in the community, and including practicing clinicians among reviewers of grant proposals. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Potential Role of Drug Onset Cues in Drug Dependence and Withdrawal: Reply to Bardo (2004), Bossert and Shaham (2004), Bouton (2004), and Stewart (2004).

In responding to commentaries (M. Bardo, see record 2004-10475-002; J. Bossert and Y. Shaham, see record 2004-10475-003; M. Bouton, see record 2004-10475-004; J. Stewart, see record 2004-10475-005) on their original article (see record 2004-10475-001), the authors agree that the basic mechanisms underlying intra-administration associations may be extensible to a much wider range of phenomena, including both other examinations of conditioned drug effects (e.g., conditioned place preference) and human psychological disorders. The authors also address the concerns of a number of the commenting authors regarding discrepancies in the literature concerning the effects of drug priming in both human and animal studies of reinstatement of drug self-administration. Finally, the authors accept and endorse the calls by several of these commenting authors for further studies required to generate additional support for their model of conditioned drug effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Modernity, postmodernity, and psychology.

Responds to the S. Greer (see record 2001-00625-016), J. D. Raskin (see record 2001-00625-017), and M. Glassman (see record 2001-00625-018) comments on the J. Martin and J. Sugarman (see record 2000-08148-003) discussion on finding the middle ground between modern and postmodern approaches to psychology. All of the critiques are responded to in turn. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comments on Ferguson's "Forty years of useless research?" (Vol. 24, no. 2, April 1983).

The authors comment on Kinglsey Ferguson's "Forty Years of Useless Research?" (see record 2007-03833-001). Ferguson has raised an important issue for all readers to consider. He has taken a courageous stand and is attempting to stimulate us, as he does his undergraduate students, to explore nondefensively where we stand as a science and what we need to do to improve the quality of our work. It it is only through the respectful marriage of the art and science of psychology that our field will prosper. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Once again, the origins of sex differences.

Responds to comments by B. X. Friedman et al (see record 2000-05933-013), D. T. Kenrick and N. Li (see record 2000-05933-014), and E. Kleyman (see record 2000-05933-015) on the article by A. H. Eagly and W. Wood (see record 1999-05337-002) which examined the origins of sex differences in human behavior. Eagly and Wood argued that social structural theory can explain the origin of psychological sex differences. In the present article, Eagly and Wood defend their original article against criticism made by the aforementioned authors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)