Electrophysiologic significance of leftward QRS axis deviation in bifascicular and trifascicular blocks

The trustworthiness of meta-analysis, a set of techniques used to quantitatively combine results from different studies, has recently been questioned. Problems with meta-analysis stem from bias in selecting studies to include in a meta-analysis and from combining study results when it is inappropriate to do so. Simple graphical techniques address these problems but are infrequently applied. Funnel plots display the relationship of effect size versus sample size and help determine whether there is likely to have been selection bias in including studies in the meta-analysis. The L'Abbé plot displays the outcomes in both the treatment and control groups of included studies and helps to decide whether the studies are too heterogeneous to appropriately combine into a single measure of effect.     

Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods.

The efficacy of the Hedges and colleagues, Rosenthal-Rubin, and Hunter-Schmidt methods for combining correlation coefficients was tested for cases in which population effect sizes were both fixed and variable. After a brief tutorial on these meta-analytic methods, the author presents 2 Monte Carlo simulations that compare these methods for cases in which the number of studies in the meta-analysis and the average sample size of studies were varied. In the fixed case the methods produced comparable estimates of the average effect size; however, the Hunter-Schmidt method failed to control the Type I error rate for the associated significance tests. In the variable case, for both the Hedges and colleagues and Hunter-Schmidt methods, Type I error rates were not controlled for meta-analyses including 15 or fewer studies and the probability of detecting small effects was less than .3. Some practical recommendations are made about the use of meta-analysis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An investment appraisal approach to clinical trial design

In this paper, a general investment appraisal model is presented which shows how pharmaceutical companies could take profit considerations into account when making decisions about the design of randomized controlled trials. A general model is presented based on the net present value method of investment appraisal. The approach is illustrated with a hypothetical example which shows how optimal (net present value maximizing) designs can be determined based on choices about sample size and endpoint measurement. The method could be extended to accommodate considerations about other trial design features, and could be used to determine a portfolio of studies which maximizes the expected return on a given development or trial budget. Furthermore, the approach could be used by pharmaceutical companies to evaluate the incremental costs and benefits of incorporating non-clinical objectives into trials, such as quality of life research and economic evaluation studies. A number of practical difficulties would need to be overcome to utilize the approach. Directions for further research are therefore highlighted centred on the key components of the model: a trial cost function, a product demand function, innovation diffusion processes and Bayesian approaches to trial design.     

A note on the sampling variance of the mean uncorrected correlation in meta-analysis and validity generalization.

Compares the accuracy of several formulas for the standard error of the mean uncorrected correlation in meta-analytic and validity generalization studies. The effect of computing the mean correlation by weighting the correlation in each study by its sample size is also studied. On the basis of formal analysis and simulation studies, it is concluded that the common formula for the sampling variance of the mean correlation, Vr ?=?Vr/K where K is the number of studies in the meta-analysis, gives reasonably accurate results. This formula gives accurate results even when sample sizes and ρs are unequal and regardless of whether or not the statistical artifacts vary from study to study. It is also shown that using sample-size weighting may result in underestimation of the standard error of the mean uncorrected correlation when there are outlier sample sizes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Overcoming the limitations of current meta-analysis of randomised controlled trials

For a meta-analysis to give definitive information, it should meet at least the minimum standards that would be expected of a well-designed, adequately powered, and carefully conducted randomised controlled trial. These minimum standards include both qualitative characteristics--a prospective protocol, comparable definitions of key outcomes, quality control of data, and inclusion of all patients from all trials in the final analysis--and quantitative standards--an assessment of whether the total sample is large enough to provide reliable results and the use of appropriate statistical monitoring guidelines to indicate when the results of the accumulating data of a meta-analysis are conclusive. We believe that rigorous meta-analyses undertaken according to these principles will lead to more reliable evidence about the efficacy and safety of interventions than either retrospective meta-analysis or individual trials.     

Update on immunotherapy for recurrent pregnancy loss

Results of recent, randomized, placebo-controlled clinical trials have raised questions about the efficacy of immunotherapy for recurrent spontaneous abortion (RSA). Most of the clinical trials have shown a 70% successful pregnancy rate with immunotherapy. The controversy comes from variations in success rates in the control populations, which have ranged from 29% to 76%. Explanations for these variations includes small sample sizes and heterogeneity by the populations studied as well as cointervention by the placebo. A meta-analysis has been proposed to investigate these explanations. Because the trials have largely used husband's leukocytes for immunization, alternative forms of immunotherapy have been sought. Two treatments that have been proposed but have not completed testing a randomized, placebo-controlled trials are intravenous immunoglobulin (IVIG) and immunization with seminal plasma. A safe and efficacious method is needed to treat recurrent spontaneous abortion; it is hoped that results of proposed studies will answer this controversy.     

On seeking moderator variables in the meta-analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error.

A series of Monte Carlo computer simulations was conducted to investigate (a) the likelihood that meta-analysis will detect true differences in effect sizes rather than attributing differences to methodological artifact and (b) the likelihood that meta-analysis will suggest the presence of moderator variables when in fact differences in effect sizes are due to methodological artifact. The simulations varied the magnitude of the true population differences between correlations, the number of studies included in the meta-analysis, and the average sample size. Simulations were run both correcting and not correcting for measurement error. The power of 3 indices—the Schmidt-Hunter ratio of expected to observed variance, the Callender-Osburn procedure, and a chi-square test—to detect true differences was investigated. Results show that small true differences were not detected regardless of sample size and the number of studies and that moderate true differences were not detected with small numbers of studies or small sample sizes. Hence, there is a need for caution in attributing observed variation across studies to artifact. (9 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Decision rules for selecting effect sizes in meta-analysis: A review and reanalysis of psychotherapy outcome studies.

This study deals with some of the judgmental factors involved in selecting effect sizes from within the studies that enter a meta-analysis. Particular attention is paid to the conceptual redundancy rule that Smith, Glass, and Miller (1980) used in their study of the effectiveness of psychotherapy for deciding which effect sizes should and should not be counted in determining an overall effect size. Data from a random sample of 25 studies from Smith et al.'s (1980) population of psychotherapy outcome studies were first recoded and then reanalyzed meta-analytically. Using the conceptual redundancy rule, three coders independently coded effect sizes and identified more than twice as many of them per study as did Smith et al. Moreover, the treatment effect estimates associated with this larger sample of effects ranged between .30 and .50, about half the size claimed by Smith et al. Analyses of other rules for selecting effect sizes showed that average effect estimates also varied with these rules. Such results indicate that the average effect estimates derived from meta-analyses may depend heavily on judgmental factors that enter into how effect sizes are selected within each of the individual studies considered relevant to a meta-analysis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analysis of multiple outcomes by regression with random effects

Earlier work showed how to perform fixed-effects meta-analysis of studies or trials when each provides results on more than one outcome per patient and these multiple outcomes are correlated. That fixed-effects generalized-least-squares approach analyzes the multiple outcomes jointly within a single model, and it can include covariates, such as duration of therapy or quality of trial, that may explain observed heterogeneity of results among the trials. Sometimes the covariates explain all the heterogeneity, and the fixed-effects regression model is appropriate. However, unexplained heterogeneity may often remain, even after taking into account known or suspected covariates. Because fixed-effects models do not make allowance for this remaining unexplained heterogeneity, the potential exists for bias in estimated coefficients, standard errors and p-values. We propose two random-effects approaches for the regression meta-analysis of multiple correlated outcomes. We compare their use with fixed-effects models and with separate-outcomes models in a meta-analysis of periodontal clinical trials. A simulation study shows the advantages of the random-effects approach. These methods also facilitate meta-analysis of trials that compare more than two treatments.     

Assignment methods in experimentation: When do nonrandomized experiments approximate answers from randomized experiments?

This meta-analysis compares effect size estimates from 51 randomized experiments to those from 47 nonrandomized experiments. These experiments were drawn from published and unpublished studies of Scholastic Aptitude Test coaching, ability grouping of students within classrooms, presurgical education of patients to improve postsurgical outcome, and drug abuse prevention with juveniles. The raw results suggest that the two kinds of experiments yield very different answers. But when studies are equated for crucial features (which is not always possible), nonrandomized experiments can yield a reasonably accurate effect size in comparison with randomized designs. Crucial design features include the activity level of the intervention given the control group, pretest effect size, selection and attrition levels, and the accuracy of the effect-size estimation method. Implications of these results for the conduct of meta-analysis and for the design of good nonrandomized experiments are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interim analyses and early termination of clinical trials

The group sequential tests (GST) are appropriate for performing interim analyses in clinical trials. Various GST are reviewed and compared in this paper in terms of the expected sample size, the maximum sample size, and other practical aspects. Also discussed are the p-values of the significant differences for GST. Common problems and difficulties of using GST in practice are examined. One problem is difficulties associated with the delayed data accumulated after a trial is terminated at an interim test. The GST with O'Brien-Fleming type of boundaries are found to be safe in dealing with delayed observations. Possible approaches for performing futility analyses are illustrated with examples. It is recommended that futility analysis with GST be built into the design of large clinical trials.     

A prospectively planned cumulative meta-analysis applied to a series of concurrent clinical trials

Sequential designs are now a familiar part of clinical trial methodology. In particular, the triangular test has been used in several individual studies. Methods of combining studies are also well-known from the literature on meta-analysis. However, the combination of the two approaches is new. Consider the situation where a series of studies is to be conducted, following broadly similar protocols comparing a new treatment with a control treatment. In order to obtain an answer as quickly as possible to an efficacy or safety question it may be desirable to perform a cumulative meta-analysis on one particular variable. This could, for example, be the primary efficacy variable, an expensive assessment conducted in only a subgroup of patients, or a serious side-effect. To allow for the size of the treatment difference varying from study to study we might wish to provide a global estimate. Hence a random effects combined analysis, within a sequential framework, would appear to be appropriate. A methodology which utilizes efficient score statistics and Fisher's information is presented. Simulations show that the proposed methodology will achieve the specified error probabilities with reasonable accuracy provided that any random effect is relatively small. Ignoring random effects when they are present can lead to inaccuracies. A simulated example illustrates a number of practical issues.     

Borrowing strength from external trials in a meta-analysis

There exists a variety of situations in which a random effects meta-analysis might be undertaken using a small number of clinical trials. A problem associated with small meta-analyses is estimating the heterogeneity between trials. To overcome this problem, information from other related studies may be incorporated into the meta-analysis. A Bayesian approach to this problem is presented using data from previous meta-analyses in the same therapeutic area to formulate a prior distribution for the heterogeneity. The treatment difference parameters are given non-informative priors. Further, related trials which compare one or other of the treatments of interest with a common third treatment are included in the model to improve inference on both the heterogeneity and the treatment difference. Two approaches to estimating relative efficacy are considered, namely a general parametric approach and a method explicit to binary data. The methodology is illustrated using data from 26 clinical trials which investigate the prevention of cirrhosis using beta-blockers and sclerotherapy. Both sources of external information lead to more precise posterior distributions for all parameters, in particular that representing heterogeneity.     

How interchangeable are different estimators of effect size?

The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Combining studies using effect sizes and quality scores: application to bone loss in postmenopausal women

This article presents a random effects model that uses effect sizes (ES) and quality scores to integrate results from investigations. An empirical example is given with data obtained from a meta-analysis on the effectiveness of physical activity in the prevention of bone loss in healthy postmenopausal women. A Medline search was performed to locate relevant studies published in French or English between January 1966 and May 1996. Three independent reviewers extracted data from studies. Effect sizes were calculated according to the method of Hedges and Olkin. A modified version of Chalmers' scale was utilized to calculate quality scores. DerSimonian and Laird's method with incorporation of the quality scores was used to estimate the overall effect size. Quality scores and the inverse of the variances were included as weights when combining studies. The overall estimate and standard error (SE) of the effect of physical activity on spinal bone mineral density loss in healthy postmenopausal women was ESoverall = 0.4263 (1.1361). When compared to other meta-analysis methods such as the fixed effects model and the model of DerSimonian and Laird without the quality score (DL), the new model generated comparable estimators (fixed effects model's ESoverall (SE) = 1.2724 (0.0139), DLs ESoverall (SE) = 0.3958 (1.2370)). Due to the heterogeneity that existed between studies, a random effects model was more appropriate then a fixed effects model. However, it resulted in wider confidence intervals, as expected. It was shown empirically that the model using quality scores generated narrower confidence intervals than the model of DL alone. The inclusion of covariates such as quality scores in meta-analyses permits the quantification of the variation between studies.     

A likelihood approach to meta-analysis with random effects

In a meta-analysis of a set of clinical trials, a crucial but problematic component is providing an estimate and confidence interval for the overall treatment effect theta. Since in the presence of heterogeneity a fixed effect approach yields an artificially narrow confidence interval for theta, the random effects method of DerSimonian and Laird, which incorporates a moment estimator of the between-trial components of variance sigma B2, has been advocated. With the additional distributional assumptions of normality, a confidence interval for theta may be obtained. However, this method does not provide a confidence interval for sigma B2, nor a confidence interval for theta which takes account of the fact that sigma B2 has to be estimated from the data. We show how a likelihood based method can be used to overcome these problems, and use profile likelihoods to construct likelihood based confidence intervals. This approach yields an appropriately widened confidence interval compared with the standard random effects method. Examples of application to a published meta-analysis and a multicentre clinical trial are discussed. It is concluded that likelihood based methods are preferred to the standard method in undertaking random effects meta-analysis when the value of sigma B2 has an important effect on the overall estimated treatment effect.     

Meta-analytic interval estimation for standardized and unstandardized mean differences.

The fixed-effects (FE) meta-analytic confidence intervals for unstandardized and standardized mean differences are based on an unrealistic assumption of effect-size homogeneity and perform poorly when this assumption is violated. The random-effects (RE) meta-analytic confidence intervals are based on an unrealistic assumption that the selected studies represent a random sample from a large superpopulation of studies. The RE approach cannot be justified in typical meta-analysis applications in which studies are nonrandomly selected. New FE meta-analytic confidence intervals for unstandardized and standardized mean differences are proposed that are easy to compute and perform properly under effect-size heterogeneity and nonrandomly selected studies. The proposed meta-analytic confidence intervals may be used to combine unstandardized or standardized mean differences from studies having either independent samples or dependent samples and may also be used to integrate results from previous studies into a new study. An alternative approach to assessing effect-size heterogeneity is presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Limitations of the dodo bird verdict and the role of clinical trials in psychotherapy research: Comment on Wampold et al. (1997).

B. E. Wampold et al.'s (1997) meta-analysis provides a useful and methodologically sophisticated summary of the results of comparative psychotherapy outcome research. Despite its strengths, some limitations of the meta-analysis that may have biased the results against finding differences between treatments are pointed out in this article. In addition, the types of treatments and patient populations to which the results can be generalized are clarified through an analysis of the studies contained within the meta-analysis. The importance of exceptions to the Dodo bird verdict is emphasized. Disagreements with Wampold et al. on the implications of the their meta-analysis for research and practice, in particular the role of clinical trials in psychotherapy research and the need for identifying treatments that are "empirically supported," are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multi-centre trial analysis revisited

Analyses of multi-centre trials must consider the effects of the individual centres and the possibility of non-constancy of treatment effect differences among centres. This usually means an ANOVA with terms for centres, treatments, and centre x treatment interactions in practice, at least in the U.S.A. Empirical and conventional Bayes methods provide attractive alternatives to conventional ANOVAs for analysing and reporting the findings from multi-centre trials and do not require more restrictive assumptions than the ANOVA approach. These approaches require regarding the centre effects as random instead of fixed, a view which often will reasonably describe outcomes of clinical trials in spite of the fact that the individual centres certainly do not comprise a random sample of all possible centres. The components of these approaches are well understood and have been employed in related applications such as meta-analysis. Combining them in a way that makes their application to routine multi-centre trial analysis relatively straightforward does not appear to have been described previously, and is what forms the topic of this paper. The empirical Bayes approach leads to useful graphical displays, including one with the data superimposed on probability contours of the joint distribution of the individual centre means and standard deviations, which provides a handy way to identify possible outliers. Covariates can be incorporated without difficulty. The Bayes approach, implemented with Gibbs sampling, provides a convenient way to construct posterior and predictive distributions for a variety of useful statistics. We compare the result of empirical and conventional Bayes analyses with the result of fixed and mixed model ANOVAs applied to data from a multi-centre trial.     

Definitive or exploratory periodontal experimentation: an overview of the literature

Definitive and exploratory randomized controlled trials (RCTs) have different goals as well as different design and analysis characteristics. The goal of definitive studies is to provide unequivocal evidence of a treatment's tangible benefit to the patient; a pre-trial-specified hypothesis is tested by use of a pre-trial-specified method. The goal of exploratory studies is to elucidate biological treatment mechanisms, to identify promising treatments, and to generate hypotheses for definitive studies: multiple hypotheses are evaluated to extract as much information from the data as possible. The purposes of this study were: (1) to survey selected design and analysis characteristics of randomized controlled periodontal trials published between 1988 and 1992 (n = 86), and (2) to classify trials as exploratory or definitive studies. The periodontal RCTs surveyed were typical of exploratory studies whose primary goal was to elucidate biological treatment mechanisms. Trial reports indicated the testing of multiple hypotheses (> or = 6 hypothesis tests in 70 of the 86 trials) on a variety of biological markers (86 out of 86 trials). The sample size (< or = 30 subjects in 67 out of 86 trials), duration (< or = 6 months in 65 out of 86 trials), and design and analysis characteristics (e.g., an absence of masking in 57 out of 86 trials) were also typical of exploratory studies which strive to obtain quick answers (short duration) at a low cost (small sample size; accept bias for increased efficiency and a lower cost). No definitive trials were identified. Promising, biologically active, treatments identified in exploratory trials should be evaluated in definitive studies where the primary goal is the procurement of unequivocal evidence of a treatment's tangible benefit to the patient. The costs and benefits of conducting definitive periodontal RCTs to provide such evidence should be investigated.