"How interchangeable are different estimators of effect size?": Correction to Ray and Shadish (1996).

Reports an error in the original article by J. W. Ray and W. R. Shadish (Journal of Consulting and Clinical Psychology, 1996[Dec], Vol 64(6), 1316–1325). On page 1325, a correction is made to column 1, lines 25–26. (The following abstract of this article originally appeared in record 1996-07086-021). The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The specific ingredients are in the match: Comments on Ahn and Wampold (2001).

H. Ahn and B. E. Wampold (see record 2001-07409-001) reported results of a meta-analysis that included 27 studies conducted between 1990 and 1999. They concluded that the effectiveness of psychological interventions is probably due to general factors, such as the therapeutic alliance, rather than treatment specificity. However, the primary studies included in the meta-analysis evaluated interventions in the abstract, ignoring client and therapist variables that may affect treatment outcome. This abstraction limits the conclusions that can be drawn from the meta-analysis. Issues of validity and bias concerning participant selection and assessment in the primary studies are discussed. It is concluded that further progress will come from evaluation of the client-therapist-intervention match. Therapist and client variables that may facilitate a positive match are described, as are implications for research and training. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reply to comments on Lipsey and Wilson (1993).

Replies to comments by H. J. Eysenck, D. Sohn, and H. Cooper et al (see PA, Vol 82:22192, 22208, and 22189, respectively) regarding Lipsey and Wilson's (see record 1994-18340-001) meta-analysis of meta-analyses. Both Sohn and Eysenck seem to be concerned with how theory is developed, but they may be confused about the role of meta-analysis in that process. It is argued that the purpose of meta-analysis is to develop generalizations about research findings across different studies. The extension of the analysis by Cooper et al reminds researchers that meta-analysis can be subject to error and bias. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Behavior therapy versus placebo: Comment on Bowers and Clum's meta-analysis.

This article evaluates a meta-analysis of 69 studies of behavior therapy versus placebos reported by T. G. Bowers and G. A. Clum (see record 1989-16477-001). The use of meta-analysis for the evaluation of therapeutic outcome is criticized. An analysis of the subset of studies in the original article dealing with neurotic patients indicates that none of the studies provided evidence for the benefits of behavior therapy that meet the criteria of significant improvement on a heteromethod set of outcome measures that is maintained at follow-up assessment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

How interchangeable are different estimators of effect size?

The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analysis of cognitive gender differences: A comment on an analysis by Rosenthal and Rubin.

Extends both the logic and the statistical procedures used in a recent analysis by R. Rosenthal and D. B. Rubin (see record 1983-02381-001) of data on gender differences in cognitive abilities reported in a meta-analysis by J. S. Hyde (see record 1982-07673-001). The logic of a "model fitting" approach to meta-analysis is described. Relevant statistical procedures and goodness-of-fit tests are illustrated. It is concluded that sex differences are changing over time and that essentially all of the variability in the sex differences reported by different studies can be explained as a function of the publication date and the selectivity of the sampling plan used in the studies. (10 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Behavior therapy better than placebo treatments: Fact or artifact?

N. Brody's (see record 1990-12934-001) comment on our meta-analysis (T. G. Bowers and G. A. Clum; see record 1989-16477-001) seems to suggest that the efficacy of behavior therapy has not been established relative to placebo control conditions, especially for "neurotic conditions." The comments appear directed at defending an earlier meta-analysis (Priouleau, Murdock, & Brody, 1983) that concluded that psychotherapy was not more efficacious than a placebo control. We agree with Brody regarding the need for increased use of the heteromethod approach and longer follow-up for psychotherapy studies. However, we do not agree that the 10 studies Brody selected do not support the effectiveness of behavior therapy relative to placebo controls. Although a set of 10 studies is probably too small to allow robust conclusions, we noted a median effect size of .63 for those studies of neurotic conditions, relative to a placebo control. These results were very similar to our overall findings from 69 studies. Furthermore, available follow-up data suggest moderate effect sizes exist for those studies. We also comment on the existence of Type I and Type II errors of inference in reviews and meta-analyses. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analyzing meta-analysis: Potential problems, an unsuccessful replication, and evaluation criteria.

Used the criteria reported by D. E. Terpstra (see record 1982-04524-001) in his study of methodological vigor in organization development evaluation research to select 90 studies, 1965–1980, for inclusion in a replication. Using the published coding scheme, the authors independently scored the 1st half-period, 1965–1972. Results show no evidence of positive-findings bias in organization development evaluation research, failing to support Terpstra's conclusion. There were differences in sample size and difficulties in coding variables. It is suggested that without a clear distinction between mixed and uniform, the coding of the research became a matter of subjective judgment. Two key decision areas of meta-analysis that may be affected by subjective biases are the coding of results and the decision rules used to code study characteristics. After a discussion of potential meta-analysis problems, 14 criteria for evaluating meta-analysis research are given. Standards include using a theoretical model, identifying the domain, including all publicly available studies, publishing a list of studies used, providing documentation of the theoretically based coding scheme, using multiple raters, and limiting generalization to the domain specified. (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Is implicit sequence learning impaired in Parkinson's disease? A meta-analysis.

The aim of the present study was to examine impairment of implicit learning in Parkinson's disease (PD) by means of a meta-analysis of studies that used the serial reaction time (SRT) task. The authors performed a systematic review and meta-analysis of published journal articles (1987-2005) that used the SRT task with patients with PD. The principal outcome measures used to compare studies were (a) the difference in reaction time between the last block of ordered sequence trials and the randomized block for PD and control participants and (b) fixed and random effects pooled estimates by the inverse weighting method. Six studies, including 67 patients with PD, met the inclusion criteria. The meta-analysis showed that implicit learning was impaired in PD, relative to healthy controls, with a standardized mean difference of 0.73 (95% confidence interval = 0.38, 1.07). Implicit sequence learning appears to be impaired in patients with PD. Some common methodological weaknesses and limitations in the reporting of statistical data are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Bactericidal eradication from carious lesions of prepared abutments by an antibacterial temporary cement

Tyrosine hydroxylase (TH) is a candidate gene extensively explored in several association studies of bipolar disorder (BD). However, because of conflicting results of independent studies and low statistical power of individual studies to detect small differences between cases and controls, reliable conclusions are difficult to formulate. A method to obtain more reliable conclusions about the involvement of the TH locus in the etiology of BD is meta-analysis. We undertook a meta-analysis of studies that investigated the association between BD and TH genetic markers. The studies were identified by means of computerized searches of several databases, and the scanning of review articles and the reference lists of the primary articles identified. More than 60 publications were reviewed, and 9 relevant articles were included in this meta-analysis, with an overall sample of 1,069 subjects (547 cases and 522 normal controls). The overall odds ratio (and confidence interval) based on combining the results of the studies was 1.02 (0.68-1.54). Test of the null hypothesis that the mean log odds ratio equals zero (chi2 = 0.11; 5 df; P > 0.05) indicated that there was no overall association between bipolar disorder and tyrosine hydroxylase.     

(Mis-) Direction of effect sizes.

Comments on an article by Dixon et al. (see record 2007-06671-001) regarding the effect sizes they presented in their meta-analysis of psychological interventions for arthritis pain management. The author of this comment claims that some of the individual effect sizes that they presented are erroneous and have therefore undermined their cumulative effect size estimates. After examining findings from other studies, he concludes that the Dixon et al. meta-analysis reports cumulative effect sizes (Hedge’s g) that overestimate the effects of psychological treatments upon arthritis pain. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An empirical and meta-analytic evaluation of the phoneme identification task.

Recent studies that used W. F. Ganong's (see record 1981-07020-001) identification task have produced discrepant results. The present study sought to resolve these discrepancies by examining the influence of methodological factors on phoneme identification and differences in data analysis techniques. Three factors were examined across 2 experiments: position of target phoneme, phonetic contrast, and 2 task conditions in which stimulus quality (S/N ratio) or cognitive load varied. A meta-analysis was then performed on the results from all identification studies, including the present one, in an effort to obtain additional insight on factors that influence the task. The experiments and meta-analysis identified the importance of several methodological factors affecting identification, most notably position of the target phoneme. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Intimate partner aggression—what have we learned? Comment on Archer (2000).

This commentary on J. Archer (see record 2000-15524-001) identifies limitations at the level of the primary data, the formal meta-analysis, and the interpretations of the results. Highlighted are concerns with the conceptual dichotomy that is the foundation of the analysis, how aggression was conceptualized and defined, and the methodological problems in the studies included in the database that were not neutralized by the meta-analysis. These include inadequate measurement of contextual factors and injury outcomes, scaling issues, and sampling concerns. The authors question the degree to which the field is advanced by this meta-analysis when the results are placed in the context of these limitations. Following American Association for the Advancement of Science directives (I. Lerch, 1999), the authors believe that inadequate attention was paid to the policy implications of the conclusions raising the potential to undermine societal efforts to eradicate violence against women. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analysis of outcome studies of Keller's personalized system of instruction.

Suggests that meta-analysis, the application of statistical methods to results from a large collection of individual studies, may prove useful to social scientists trying to draw reliable and general conclusions from a diverse and voluminous literature. This article describes a meta-analysis of 75 comparative studies of an innovative method of college teaching, F. S. Keller's (see record 1969-04532-001) personalized system of instruction (PSI). The analysis establishes that PSI generally produces superior student achievement, less variation in achievement, and higher student ratings in college courses, but does not affect course withdrawal or student study time in these courses. The analysis also shows that PSI's superiority can be demonstrated in a variety of course settings with a number of different research designs. Certain settings and research designs, however, produce especially sharp differences between PSI and conventional courses. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relative benefits of meta-analysis conducted with individual participant data versus aggregated data.

The authors describe the relative benefits of conducting meta-analyses with (a) individual participant data (IPD) gathered from the constituent studies and (b) aggregated data (AD), or the group-level statistics (in particular, effect sizes) that appear in reports of a study’s results. Given that both IPD and AD are equally available, meta-analysis of IPD is superior to meta-analysis of AD. IPD meta-analysis permits synthesists to perform subgroup analyses not conducted by the original collectors of the data, to check the data and analyses in the original studies, to add new information to the data sets, and to use different statistical methods. However, the cost of IPD meta-analysis and the lack of available IPD data sets suggest that the best strategy currently available is to use both approaches in a complementary fashion such that the first step in conducting an IPD meta-analysis would be to conduct an AD meta-analysis. Regardless of whether a meta-analysis is conducted with IPD or AD, synthesists must remain vigilant in how they interpret their results. They must avoid ecological fallacies, Simpson’s paradox, and interpretation of synthesis-generated evidence as supporting causal inferences. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

In defense of Social Impact Theory: Comment on Mullen.

Disputes B. Mullen's (see record 1985-28034-001) conclusions regarding studies involving the strength and immediacy variables of social impact theory. It is argued that the conceptual analysis of this theory is faulty, that the categorization scheme used in the meta-analysis is misleading, and that several bodies of literature pertinent to this analysis were systematically excluded. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

病例对照Meta分析显示男性乳腺癌与女性乳腺癌预后相同

Objective: Male breast cancer (MBC) is a rare disease, and studies to compare the prognosis between the MBC and female breast cancer (FBC) available now were case-control studies and population-based studies, however, with controversial outcomes. To further investigate whether sex affects the prognosis of breast cancer, we performed a meta-analysis based on all the peered papers. Methods: The PubMed database was screened. Subsequently, the hazard ratios (HRs) for overall survival (OS) and disease special survival (DSS), were pooled using the Review Manager 5.0 package. Results: This study comprised of 254 MBC and 733 FBC patients enrolled in six case-control studies. There was no statistically significant heterogeneity among the trials for either outcome variable. The pooled HR showed that there was no significant difference in 5-year or 10-year DSS (HR: 1.25, 95% CI: 0.76-2.08; HR: 1.04, 95% CI: 0.66-1.59) and 5-year or 10-year OS (HR: 0.76, 95%CI: 0.51-1.14; HR: 0.74, 95% CI: 0.54-1.02) between MBCs and FBCs. Conclusion: MBC and FBC with similar prognosis are found in this meta-analysis based on only case-control studies which matched main potential prognostic factors such as age, clinical stage, and time at diagnosis.     

Experimental effects and person effects in delay of gratification.

Elaborates on findings included in W. Mischel's (see record 1985-06679-001) article regarding the effect of personality factors and experimental manipulations on delayed gratification by presenting a meta-analysis of experimental studies related to delay of gratification and by suggesting that broad range effect sizes are larger and more important than they are generally considered to be. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of strength and immediacy in group contexts: Reply to Jackson.

Responds to J. M. Jackson's (see record 1986-19799-001) comments on the present author's (see record 1985-28034-001) meta-analysis of the effects of strength and immediacy in group contexts. It is argued that the conceptual analysis of social impact theory was unflattering but accurate, that the categorization scheme used in the meta-analysis was straightforward and defensible, and that the inclusion criteria for the meta-analysis provided the opportunity for an unbiased test of social impact theory's implications. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"An exercise in mega-silliness": Reply.

Replies to H. J. Eysenck's (1978) criticisms of M. L. Smith and G. V. Glass's (see record 1978-10341-001) meta-analysis of psychotherapy outcome studies. Eysenck's rejection of any evidence supporting the effectiveness of psychotherapy is disputed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)