Borrowing strength from external trials in a meta-analysis

There exists a variety of situations in which a random effects meta-analysis might be undertaken using a small number of clinical trials. A problem associated with small meta-analyses is estimating the heterogeneity between trials. To overcome this problem, information from other related studies may be incorporated into the meta-analysis. A Bayesian approach to this problem is presented using data from previous meta-analyses in the same therapeutic area to formulate a prior distribution for the heterogeneity. The treatment difference parameters are given non-informative priors. Further, related trials which compare one or other of the treatments of interest with a common third treatment are included in the model to improve inference on both the heterogeneity and the treatment difference. Two approaches to estimating relative efficacy are considered, namely a general parametric approach and a method explicit to binary data. The methodology is illustrated using data from 26 clinical trials which investigate the prevention of cirrhosis using beta-blockers and sclerotherapy. Both sources of external information lead to more precise posterior distributions for all parameters, in particular that representing heterogeneity.     

Predictors of posttraumatic stress disorder and symptoms in adults: A meta-analysis.

A review of 2,647 studies of posttraumatic stress disorder (PTSD) yielded 476 potential candidates for a meta-analysis of predictors of PTSD or of its symptoms. From these, 68 studies met criteria for inclusion in a meta-analysis of 7 predictors: (a) prior trauma, (b) prior psychological adjustment, (c) family history of psychopathology, (d) perceived life threat during the trauma, (e) posttrauma social support, (f) peritraumatic emotional responses, and (g) peritraumatic dissociation. All yielded significant effect sizes, with family history, prior trauma, and prior adjustment the smallest (weighted r=.17) and peritraumatic dissociation the largest (weighted r=.35). The results suggest that peritraumatic psychological processes, not prior characteristics, are the strongest predictors of PTSD. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Behavior therapy better than placebo treatments: Fact or artifact?

N. Brody's (see record 1990-12934-001) comment on our meta-analysis (T. G. Bowers and G. A. Clum; see record 1989-16477-001) seems to suggest that the efficacy of behavior therapy has not been established relative to placebo control conditions, especially for "neurotic conditions." The comments appear directed at defending an earlier meta-analysis (Priouleau, Murdock, & Brody, 1983) that concluded that psychotherapy was not more efficacious than a placebo control. We agree with Brody regarding the need for increased use of the heteromethod approach and longer follow-up for psychotherapy studies. However, we do not agree that the 10 studies Brody selected do not support the effectiveness of behavior therapy relative to placebo controls. Although a set of 10 studies is probably too small to allow robust conclusions, we noted a median effect size of .63 for those studies of neurotic conditions, relative to a placebo control. These results were very similar to our overall findings from 69 studies. Furthermore, available follow-up data suggest moderate effect sizes exist for those studies. We also comment on the existence of Type I and Type II errors of inference in reviews and meta-analyses. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analysis of condom effectiveness in reducing sexually transmitted HIV

Before condoms can be considered as a prophylaxis for sexually transmitted human immunodeficiency virus (HIV), their efficacy must be considered. This paper reviews evidence on condom effectiveness in reducing the risk of heterosexually transmitted human HIV. A meta-analysis conducted on data from in vivo studies of HIV discordant sexual partners is used to estimate the protective effect of condoms. Although contraceptive research indicates that condoms are 87% effective in preventing pregnancy, results of HIV transmission studies indicate that condoms may reduce risk of HIV infection by approximately 69%. Thus, efficacy may be much lower than commonly assumed, although results should be viewed tentatively due to design limitations in the original studies.     

Update on immunotherapy for recurrent pregnancy loss

Results of recent, randomized, placebo-controlled clinical trials have raised questions about the efficacy of immunotherapy for recurrent spontaneous abortion (RSA). Most of the clinical trials have shown a 70% successful pregnancy rate with immunotherapy. The controversy comes from variations in success rates in the control populations, which have ranged from 29% to 76%. Explanations for these variations includes small sample sizes and heterogeneity by the populations studied as well as cointervention by the placebo. A meta-analysis has been proposed to investigate these explanations. Because the trials have largely used husband's leukocytes for immunization, alternative forms of immunotherapy have been sought. Two treatments that have been proposed but have not completed testing a randomized, placebo-controlled trials are intravenous immunoglobulin (IVIG) and immunization with seminal plasma. A safe and efficacious method is needed to treat recurrent spontaneous abortion; it is hoped that results of proposed studies will answer this controversy.     

Assessing placebo response using Bayesian hierarchical survival models

The National Institute of Mental Health (NIMH) Collaborative Study of Long-Term Maintenance Drug Therapy in Recurrent Affective Illness was a multicenter randomized controlled clinical trial designed to determine the efficacy of a pharmacotherapy for the prevention of the recurrence of unipolar affective disorders. The outcome of interest in this study was the time until the recurrence of a depressive episode. The data show much heterogeneity between centers for the placebo group. The aim of this paper is to use Bayesian hierarchical survival models to investigate the heterogeneity of placebo effects among centers in the NIMH study. This heterogeneity is explored in terms of the marginal posterior distributions of parameters of interest and predictive distributions of future observations. The Gibbs sampling algorithm is used to approximate posterior and predictive distributions. Sensitivity of results to the assumption of a constant hazard survival distribution at the first stage of the hierarchy is examined by comparing results derived from a two component exponential mixture and a two component exponential changepoint model to the results derived from an exponential model. The second component of the mixture and changepoint models is assumed to be a surviving fraction. For each of these first stage parametric models sensitivity of results to second stage prior distributions is also examined.     

Incorporation of sequential trials into a fixed effects meta-analysis

The technique of meta-analysis provides a systematic and quantitative approach to the summary of results from a collection of similar randomized studies. Comprehensive methodology exists for analysis when all trials are of a fixed sample size design, but this is based on assumptions which are no longer valid when incorporating sequentially designed studies. In this paper the effect of incorporating one or more sequential trials into a meta-analysis otherwise consisting of fixed sample size trials is investigated. The results from simulation studies of a variety of plausible scenarios are presented. The aim is to quantify the extent to which bias is introduced. It was found that when incorporating two alternative sequential designs, the triangular test and the O'Brien and Fleming procedure, the results of a conventional meta-analysis remain accurate.     

Overcoming the limitations of current meta-analysis of randomised controlled trials

For a meta-analysis to give definitive information, it should meet at least the minimum standards that would be expected of a well-designed, adequately powered, and carefully conducted randomised controlled trial. These minimum standards include both qualitative characteristics--a prospective protocol, comparable definitions of key outcomes, quality control of data, and inclusion of all patients from all trials in the final analysis--and quantitative standards--an assessment of whether the total sample is large enough to provide reliable results and the use of appropriate statistical monitoring guidelines to indicate when the results of the accumulating data of a meta-analysis are conclusive. We believe that rigorous meta-analyses undertaken according to these principles will lead to more reliable evidence about the efficacy and safety of interventions than either retrospective meta-analysis or individual trials.     

Roaming through the methodology. VIII. Pilot studies: sense and nonsense

A pilot study is often performed before the start of a large study. Its aim is improving the methodological quality and evaluating the feasibility. The results of a pilot study are also often used to gain an impression of the efficacy of an intervention. For this purpose a pilot is absolutely unsuited. However, the estimate of the effect of an intervention in small studies such as pilot studies is determined to a large extent by chance. When conducting small studies the chance of publication bias is large. Small studies of adequate methodological quality should be published, whether the results are positive or negative, significant or non-significant, because combination in a later cumulative meta-analysis may lead to sufficient power to assess the efficacy of an experimental intervention.     

The statistical basis of public policy: a paradigm shift is overdue

The recent controversy over the increased risk of venous thrombosis with third generation oral contraceptives illustrates the public policy dilemma that can be created by relying on conventional statistical tests and estimates: case-control studies showed a significant increase in risk and forced a decision either to warn or not to warn. Conventional statistical tests are an improper basis for such decisions because they dichotomise results according to whether they are or are not significant and do not allow decision makers to take explicit account of additional evidence--for example, of biological plausibility or of biases in the studies. A Bayesian approach overcomes both these problems. A Bayesian analysis starts with a "prior" probability distribution for the value of interest (for example, a true relative risk)--based on previous knowledge--and adds the new evidence (via a model) to produce a "posterior" probability distribution. Because different experts will have different prior beliefs sensitivity analyses are important to assess the effects on the posterior distributions of these differences. Sensitivity analyses should also examine the effects of different assumptions about biases and about the model which links the data with the value of interest. One advantage of this method is that it allows such assumptions to be handled openly and explicitly. Data presented as a series of posterior probability distributions would be a much better guide to policy, reflecting the reality that degrees of belief are often continuous, not dichotomous, and often vary from one person to another in the face of inconclusive evidence.     

Efficacy of rational-emotive therapy: A quantitative analysis.

Meta-analysis of 28 controlled studies on the efficacy of rational-emotive therapy (RET) showed RET to be superior to placebo and no treatment but equally effective in comparison with other types of treatment such as combination therapies and systematic desensitization. No support was found for the view that RET with a main or balanced emphasis on behavioral techniques is more efficacious than RET with a primarily or exclusively cognitive approach. However, given the heterogeneity of the set of studies and its relatively small sample size, interpretations of results have to be made with caution, particularly because evidence was found for a relationship between study characteristics and magnitude of effect size. For the sake of meta-analysis, outcome studies should be required to report at least the means and standard deviations for all experimental groups. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Seventy-two tests of the sequential lineup superiority effect: A meta-analysis and policy discussion.

A decade ago, a meta-analysis showed that identification of a suspect from a sequential lineup versus a simultaneous lineup was more diagnostic of guilt (Steblay, Dysart, Fulero, & Lindsay, 2001). Since then, controversy and debate regarding sequential superiority has emerged. We report the results of a new meta-analysis involving 72 tests of simultaneous and sequential lineups from 23 different labs involving 13,143 participant-witnesses. The results are very similar to the 2001 results in showing that the sequential lineup is less likely to result in an identification of the suspect, but also more diagnostic of guilt than is the simultaneous lineup. An examination of the full diagnostic design dataset (27 tests that used the full simultaneous/sequential × culprit-present/culprit-absent design) showed that the average gap in correct identifications favoring the simultaneous lineup over the sequential lineup—8%—is smaller than the 15% figure obtained from the 2001 meta-analysis (and from the current full 72-test dataset). The lower error rate incurred for culprit-absent lineups with use of a sequential format remains consistent across the years, with 22% fewer errors than simultaneous lineups. A Bayesian analysis shows that the posterior probability of guilt following an identification of the suspect is higher for the sequential lineup across the entire base rate for culprit presence/absence. New ways to think about policy issues are discussed. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Extraversion and Emotional Reactivity.

Six mood induction studies and a meta-analysis were conducted to test 2 models of the extraversion-pleasant affect relation. The affect-level model suggests that extraverts should be happier than introverts in both neutral and positive mood conditions. The reactivity model posits that extraverts react particularly strongly to pleasant stimuli and that they should be happier than introverts only in positive conditions. In all studies, extraverts failed to exhibit greater emotional reactivity when pleasantness items were analyzed. When activated positive affect items were analyzed, results were mixed. The meta-analysis confirmed that there is only a slight reactivity effect overall, and this effect emerges only in activated positive affect items. Furthermore, the meta-analysis showed that the correlation in neutral conditions is strong enough to support the affect-level model. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Bayesian characterization of Hardy-Weinberg disequilibrium

A Bayesian method for determining if there are large departures from independence between pairs of alleles at a locus, Hardy-Weinberg equilibrium (HWE), is presented. We endorse the view that a population will never be exactly in HWE and that there will be occasions when there is a need for an alternative to the usual hypothesis-testing setting. Bayesian methods provide such an alternative, and our approach differs from previous Bayesian treatments in using the disequilibrium and inbreeding coefficient parameterizations. These are easily interpretable but may be less mathematically tractable than other parameterizations. We examined the posterior distributions of our parameters for evidence that departures from HWE were large. For either parameterization, when a conjugate prior was used, the prior probability for small departures was itself small, i.e., the prior was weighted against small departures from independence. We could avoid this uneven weighting by using a step prior which gave equal weighting to both small and large departures from HWE. In most cases, the Bayesian methodology makes it clear that there are not enough data to draw a conclusion.     

A meta-analysis of the antecedents and consequences of pay level satisfaction.

This study reports results from a meta-analysis of 28 correlates of pay level satisfaction involving 240 samples from 203 studies conducted over the past 35 years. Results are presented in 4 categories: primary determinants, antecedents, correlates, and outcomes of pay satisfaction. The authors controlled for pay in examining relations between correlates and pay level satisfaction, as suggested by theory and when primary studies were available to do so. The authors found support for many of the relations suggested by a theoretical model and also note some limitations in the research that has tested this model. The authors recommend changes and additions to the model and suggest additional primary research in specific areas. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Meta-analysis of multiple outcomes by regression with random effects

Earlier work showed how to perform fixed-effects meta-analysis of studies or trials when each provides results on more than one outcome per patient and these multiple outcomes are correlated. That fixed-effects generalized-least-squares approach analyzes the multiple outcomes jointly within a single model, and it can include covariates, such as duration of therapy or quality of trial, that may explain observed heterogeneity of results among the trials. Sometimes the covariates explain all the heterogeneity, and the fixed-effects regression model is appropriate. However, unexplained heterogeneity may often remain, even after taking into account known or suspected covariates. Because fixed-effects models do not make allowance for this remaining unexplained heterogeneity, the potential exists for bias in estimated coefficients, standard errors and p-values. We propose two random-effects approaches for the regression meta-analysis of multiple correlated outcomes. We compare their use with fixed-effects models and with separate-outcomes models in a meta-analysis of periodontal clinical trials. A simulation study shows the advantages of the random-effects approach. These methods also facilitate meta-analysis of trials that compare more than two treatments.     

Effectiveness of attention rehabilitation after an acquired brain injury: A meta-analysis.

The efficacy of attention rehabilitation after an acquired brain injury was examined meta-analytically. Thirty studies with a total of 359 participants met the authors' selection criteria. Studies were categorized according to whether training efficacy was evaluated by comparing pre- and posttraining scores only or included a control condition as well. Performance improved significantly (using the d+ statistic) after training in pre-post only studies but not in pre-post with control studies. Further analyses showed that specific-skills training significantly improved performance of tasks requiring attention but that the cognitive-retraining methods included in the meta-analysis did not significantly affect outcomes. These findings demonstrate that acquired deficits of attention are treatable using specific-skills training. Implications of these results for rehabilitation theory and future research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Electrophysiologic significance of leftward QRS axis deviation in bifascicular and trifascicular blocks

The trustworthiness of meta-analysis, a set of techniques used to quantitatively combine results from different studies, has recently been questioned. Problems with meta-analysis stem from bias in selecting studies to include in a meta-analysis and from combining study results when it is inappropriate to do so. Simple graphical techniques address these problems but are infrequently applied. Funnel plots display the relationship of effect size versus sample size and help determine whether there is likely to have been selection bias in including studies in the meta-analysis. The L'Abbé plot displays the outcomes in both the treatment and control groups of included studies and helps to decide whether the studies are too heterogeneous to appropriately combine into a single measure of effect.     

A meta-analysis of adolescent smoking prevention programs

OBJECTIVES: A large number of studies evaluating adolescent smoking prevention programs have been published. Systematic quantitative reviews of this literature are needed to learn what does and does not work. The present meta-analysis focuses on the efficacy of school-based programs. METHODS: Evaluations of 94 separate interventions were included in the meta-analysis. Studies were screened for methodological rigor and those with weaker methodology were segregated from those with more defensible methodology; major analyses focused on the latter. RESULTS: Behavioral effect sizes were found to be largest for interventions with a social reinforcement orientation, moderate for interventions with either a developmental or a social norms orientation, and small for interventions with the traditional rational orientation. Attitude effect sizes followed the same pattern, but knowledge effect sizes were similar across all four orientation categories. CONCLUSIONS: Because behavioral effect represents the fundamental objective of programs for prevention of adolescent tobacco use, the present results indicate that school-based programs should consider adopting interventions with a social reinforcement, social norms, or developmental orientation.