Confidence intervals for the overall effect size in random-effects meta-analysis.

One of the main objectives in meta-analysis is to estimate the overall effect size by calculating a confidence interval (CI). The usual procedure consists of assuming a standard normal distribution and a sampling variance defined as the inverse of the sum of the estimated weights of the effect sizes. But this procedure does not take into account the uncertainty due to the fact that the heterogeneity variance (τ2) and the within-study variances have to be estimated, leading to CIs that are too narrow with the consequence that the actual coverage probability is smaller than the nominal confidence level. In this article, the performances of 3 alternatives to the standard CI procedure are examined under a random-effects model and 8 different τ2 estimators to estimate the weights: the t distribution CI, the weighted variance CI (with an improved variance), and the quantile approximation method (recently proposed). The results of a Monte Carlo simulation showed that the weighted variance CI outperformed the other methods regardless of the τ2 estimator, the value of τ2, the number of studies, and the sample size. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improved confidence intervals for the difference between binomial proportions based on paired data

Existing methods for setting confidence intervals for the difference theta between binomial proportions based on paired data perform inadequately. The asymptotic method can produce limits outside the range of validity. The 'exact' conditional method can yield an interval which is effectively only one-sided. Both these methods also have poor coverage properties. Better methods are described, based on the profile likelihood obtained by conditionally maximizing the proportion of discordant pairs. A refinement (methods 5 and 6) which aligns 1-alpha with an aggregate of tail areas produces appropriate coverage properties. A computationally simpler method based on the score interval for the single proportion also performs well (method 10).     

How Meta-Analysis Increases Statistical Power.

One of the most frequently cited reasons for conducting a meta-analysis is the increase in statistical power that it affords a reviewer. This article demonstrates that fixed-effects meta-analysis increases statistical power by reducing the standard error of the weighted average effect size (T?.) and, in so doing, shrinks the confidence interval around T?.. Small confidence intervals make it more likely for reviewers to detect nonzero population effects, thereby increasing statistical power. Smaller confidence intervals also represent increased precision of the estimated population effect size. Computational examples are provided for 3 effect-size indices: d (standardized mean difference), Pearson's r, and odds ratios. Random-effects meta-analyses also may show increased statistical power and a smaller standard error of the weighted average effect size. However, the authors demonstrate that increasing the number of studies in a random-effects meta-analysis does not always increase statistical power. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analysis of zinc salts lozenges and the common cold

BACKGROUND: In the United States, the common cold has been estimated to cost more than $3.5 billion a year. Despite several randomized clinical trials, the effect of treating colds with zinc salts lozenges remains uncertain because of conflicting results. OBJECTIVE: To conduct a meta-analysis of published randomized clinical trials on the use of zinc salts lozenges in colds using a random effects model. RESULTS: Eight clinical trials of treating adults with zinc salts lozenges were identified. After excluding 2 studies that used nasal inoculation of rhinovirus, 6 trials were combined and analyzed. The summary odds ratio for the presence of any cold symptoms at 7 days was 0.50 (95% confidence interval, 0.19-1.29). CONCLUSION: Despite numerous randomized trials, the evidence for effectiveness of zinc salts lozenges in reducing the duration of common colds is still lacking.     

Combining studies using effect sizes and quality scores: application to bone loss in postmenopausal women

This article presents a random effects model that uses effect sizes (ES) and quality scores to integrate results from investigations. An empirical example is given with data obtained from a meta-analysis on the effectiveness of physical activity in the prevention of bone loss in healthy postmenopausal women. A Medline search was performed to locate relevant studies published in French or English between January 1966 and May 1996. Three independent reviewers extracted data from studies. Effect sizes were calculated according to the method of Hedges and Olkin. A modified version of Chalmers' scale was utilized to calculate quality scores. DerSimonian and Laird's method with incorporation of the quality scores was used to estimate the overall effect size. Quality scores and the inverse of the variances were included as weights when combining studies. The overall estimate and standard error (SE) of the effect of physical activity on spinal bone mineral density loss in healthy postmenopausal women was ESoverall = 0.4263 (1.1361). When compared to other meta-analysis methods such as the fixed effects model and the model of DerSimonian and Laird without the quality score (DL), the new model generated comparable estimators (fixed effects model's ESoverall (SE) = 1.2724 (0.0139), DLs ESoverall (SE) = 0.3958 (1.2370)). Due to the heterogeneity that existed between studies, a random effects model was more appropriate then a fixed effects model. However, it resulted in wider confidence intervals, as expected. It was shown empirically that the model using quality scores generated narrower confidence intervals than the model of DL alone. The inclusion of covariates such as quality scores in meta-analyses permits the quantification of the variation between studies.     

The association of respiratory symptoms and lung function with the use of gas for cooking. European Community Respiratory Health Survey

The association of respiratory symptoms and lung function with the use of gas for cooking was examined using data collected as part of the European Community Respiratory Health Survey, an international multicentre study. Associations between gas cooking and respiratory symptoms and respiratory function were assessed by logistic and multiple regression models. Tests for interaction were used to examine whether the effect of gas cooking varied between centres and, as there was evidence for this, the average effects were estimated using standard methods for random effects meta-analysis. Data from 5,561 males and 6,029 females living in 23 centres in 11 countries were analysed. There was no significant association found between respiratory symptoms and gas cooking in males. In females the association between some respiratory symptoms and gas cooking varied between centres with an overall positive association with "wheeze in the last 12 months" (odds ratio (OR) 1.24: 95% confidence interval (95% CI) 1.00-1.54) and "wheeze with breathlessness in the last 12 months" (OR 1.33: 95% CI 1.06-1.69). There was no evidence that atopy modified this association. Cooking with gas was associated with airways obstruction in both males and females although the differences failed to reach statistical significance. In some countries the use of gas for cooking is associated with respiratory symptoms suggestive of airways obstruction in females.     

Expected versus observed information in SEM with incomplete normal and nonnormal data.

Maximum likelihood is the most common estimation method in structural equation modeling. Standard errors for maximum likelihood estimates are obtained from the associated information matrix, which can be estimated from the sample using either expected or observed information. It is known that, with complete data, estimates based on observed or expected information are consistent. The situation changes with incomplete data. When the data are missing at random (MAR), standard errors based on expected information are not consistent, and observed information should be used. A less known fact is that in the presence of nonnormality, the estimated information matrix also enters the robust computations (both standard errors and the test statistic). Thus, with MAR nonnormal data, the use of the expected information matrix can potentially lead to incorrect robust computations. This article summarizes the results of 2 simulation studies that investigated the effect of using observed versus expected information estimates of standard errors and test statistics with normal and nonnormal incomplete data. Observed information is preferred across all conditions. Recommendations to researchers and software developers are outlined. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Inference for odds ratio regression models with sparse dependent data

Suppose the number of 2 x 2 tables is large relative to the average table size, and the observations within a given table are dependent, as occurs in longitudinal or family-based case-control studies. We consider fitting regression models to the odds ratios using table-level covariates. The focus is on methods to obtain valid inferences for the regression parameters beta when the dependence structure is unknown. In this setting, Liang (1985, Biometrika 72, 678-682) has shown that inference based on the noncentral hypergeometric likelihood is sensitive to misspecification of the dependence structure. In contrast, estimating functions based on the Mantel-Haenszel method yield consistent estimators of beta. We show here that, under the estimating function approach, Wald's confidence interval for beta performs well in multiplicative regression models but unfortunately has poor coverage probabilities when an additive regression model is adopted. As an alternative to Wald inference, we present a Mantel-Haenszel quasi-likelihood function based on integrating the Mantel-Haenszel estimating function. A simulation study demonstrates that, in medium-sized samples, the Mantel-Haenszel quasi-likelihood approach yields better inferences than other methods under an additive regression model and inferences comparable to Wald's method under a multiplicative model. We illustrate the use of this quasi-likelihood method in a study of the familial risk of schizophrenia.     

Controversies in meta-analysis: the case of the trials of serum cholesterol reduction

There has recently been disagreement in the literature on the results and interpretation of meta-analyses of the trials of serum cholesterol reduction, both in terms of the quantification of the effect on ischaemic heart disease and as regards the evidence of any adverse effect on other causes of death. This paper describes statistical aspects of a recent meta-analysis of these trials, and draws some more general conclusions about the methods used in meta-analysis. Tests of an overall null hypothesis are shown to have a basis clearly distinct from the more extensive assumptions needed to provide an overall estimate of effect. The fixed effect approach to estimation relies on the implausible assumption of homogeneity of treatment effects across the trials, and is therefore likely to yield confidence intervals which are too narrow and conclusions which are too dogmatic. However the conventional random effects method relies on its own set of unrealistic assumptions, and cannot be regarded as a robust solution to the problem of statistical heterogeneity. The random effects method is more usefully regarded as a type of sensitivity analysis in which the weights allocated to each study in estimating the overall effect are modified. However, rather than using a statistical model for the 'unexplained' heterogeneity, greater insight and scientific understanding of the results of a set of trials may be obtained by a careful exploration of potential sources of heterogeneity. In the context of the cholesterol trials, the heterogeneity according to the extent and duration of cholesterol reduction are of prime concern and are investigated using logistic regression. It is concluded that the long-term benefits of serum cholesterol reduction on the risk of heart disease have been seriously underestimated in some previous meta-analyses, while the evidence for adverse effects on other causes of death have been misleadingly exaggerated.     

Bactericidal eradication from carious lesions of prepared abutments by an antibacterial temporary cement

Tyrosine hydroxylase (TH) is a candidate gene extensively explored in several association studies of bipolar disorder (BD). However, because of conflicting results of independent studies and low statistical power of individual studies to detect small differences between cases and controls, reliable conclusions are difficult to formulate. A method to obtain more reliable conclusions about the involvement of the TH locus in the etiology of BD is meta-analysis. We undertook a meta-analysis of studies that investigated the association between BD and TH genetic markers. The studies were identified by means of computerized searches of several databases, and the scanning of review articles and the reference lists of the primary articles identified. More than 60 publications were reviewed, and 9 relevant articles were included in this meta-analysis, with an overall sample of 1,069 subjects (547 cases and 522 normal controls). The overall odds ratio (and confidence interval) based on combining the results of the studies was 1.02 (0.68-1.54). Test of the null hypothesis that the mean log odds ratio equals zero (chi2 = 0.11; 5 df; P > 0.05) indicated that there was no overall association between bipolar disorder and tyrosine hydroxylase.     

A multivariate mixed linear model for meta-analysis.

A multivariate mixed-effects approach for meta-analysis is presented. The approach (a) incorporates as outcomes multiple effect sizes per study; (b) allows different studies to have different subsets of effect sizes; and (c) treats each study's effect sizes as random realizations from a population of possible effect sizes. Application is illustrated via reanalysis of data from studies assessing the effects of coaching on verbal and mathematical subtests of the Scholastic Aptitude Test. Covariance components are estimated via restricted maximum likelihood (REML); inferences about regression coefficients and specific study effect sizes are based on their joint conditional distribution given the REML covariance component estimates. The approach can be implemented via now-standard software for unbalanced hierarchical data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sample Size for Multiple Regression: Obtaining Regression Coefficients That Are Accurate, Not Simply Significant.

An approach to sample size planning for multiple regression is presented that emphasizes accuracy in parameter estimation (AIPE). The AIPE approach yields precise estimates of population parameters by providing necessary sample sizes in order for the likely widths of confidence intervals to be sufficiently narrow. One AIPE method yields a sample size such that the expected width of the confidence interval around the standardized population regression coefficient is equal to the width specified. An enhanced formulation ensures, with some stipulated probability, that the width of the confidence interval will be no larger than the width specified. Issues involving standardized regression coefficients and random predictors are discussed, as are the philosophical differences between AIPE and the power analytic approaches to sample size planning. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Management of iliac occlusions with a new self-expanding endovascular stent

Randomized, prospective studies comparing BUCY to TBI conditioning regimens for allogeneic bone marrow transplantation have yielded conflicting results. We investigated the overall survival, the disease-free survival and the toxicities of BUCY vs TBI-based regimens by conducting a meta-analysis of all published, randomized, prospective trials comparing these regimens. Five studies were analyzed. We evaluated six endpoints: survival, disease-free survival, veno-occlusive disease (VOD) of the liver, acute GVHD, chronic GVHD, and interstitial pneumonitis. We combined individual study results using a random effects model. Survival and disease-free survival were better with TBI-based regimens than with BUCY, but these differences were not statistically significant (survival odds ratio 1.4, 95% confidence interval 0.9-2.2, P = 0.09; disease-free survival odds ratio 1.2, 95% confidence interval 0.7-2.1, P = 0.44). A power analysis indicated that BUCY was unlikely to have a clinically relevant survival or disease-free survival advantage. The power analysis could not exclude the possibility of such an advantage for TBI-based regimens. A significantly greater incidence of VOD occurred with BUCY (odds ratio 2.5, 95% confidence interval 1.2-5.2, P = 0.02). For the other side-effects, there were no significant differences. We concluded that TBI-based regimens cause less VOD than BUCY and are at least as good for survival and disease-free survival.     

Testing differences between nested covariance structure models: Power analysis and null hypotheses.

For comparing nested covariance structure models, the standard procedure is the likelihood ratio test of the difference in fit, where the null hypothesis is that the models fit identically in the population. A procedure for determining statistical power of this test is presented where effect size is based on a specified difference in overall fit of the models. A modification of the standard null hypothesis of zero difference in fit is proposed allowing for testing an interval hypothesis that the difference in fit between models is small, rather than zero. These developments are combined yielding a procedure for estimating power of a test of a null hypothesis of small difference in fit versus an alternative hypothesis of larger difference. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of proximal isovelocity surface area method with pressure half-time and planimetry in evaluation of mitral stenosis

OBJECTIVES: This study sought to 1) compare the accuracy of the proximal isovelocity surface area (PISA) and Doppler pressure half-time methods and planimetry for echocardiographic estimation of mitral valve area; 2) evaluate the effect of atrial fibrillation on the accuracy of the PISA method; and 3) assess factors used to correct PISA area estimates for leaflet angulation. BACKGROUND: Despite recognized limitations of traditional echocardiographic methods for estimating mitral valve area, there has been no systematic comparison with the PISA method in a single cohort. METHODS: Area estimates were obtained in patients with mitral stenosis by the Gorlin hydraulic formula, PISA and pressure half-time method in 48 patients and by planimetry in 36. Two different factors were used to correct PISA estimates for leaflet angle (theta): 1) plane-angle factor (theta/180 [theta in degrees]); and 2) solid-angle factor [1-cos(theta/2)]. RESULTS: After exclusion of patients with significant mitral regurgitation, the correlation between Gorlin and PISA areas (0.88) was significantly greater (p < 0.04) than that between Gorlin and pressure half-time (0.78) or Gorlin and planimetry (0.72). The correlation between Gorlin and PISA area estimates was lower in atrial fibrillation than sinus rhythm (0.69 vs. 0.93), but the standard error of the estimate was only slightly greater (0.24 vs. 0.19 cm2). The average ratio of the solid- to the plane-angle correction factors was approximately equal to previously reported values of the orifice contraction coefficient for tapering stenosis. CONCLUSIONS: 1) The accuracy of PISA area estimates in mitral stenosis is at least comparable to those of planimetry and pressure half-time. 2) Reasonable accuracy of the PISA method is possible in irregular rhythms. 3) A simple leaflet angle correction factor, theta/180 (theta in degrees), yields the physical orifice area because it overestimates the vena contracta area by a factor approximately equal to the contraction coefficient for a tapering stenosis.     

Meta-analysis of physiological outcomes of hospital-based infant intervention programs

A meta-analysis of 37 studies on the effects of an intervention program on physiological development of the premature infant was conducted using multidisciplinary research reported since 1979. Two homogeneous populations based on mean infant birth weight were found in which infant stimulation programs were shown to be effective in enhancing physiological development (Group I: mean birth weight = 1,260 g, mean combined effect size = 1.54, SD = 0.54, 95% confidence interval = 1.22-1.86; Group II: mean birth weight = 1,524 g, mean combined ES = 0.51, SD = 0.30, 95% confidence interval = 0.39-0.63). The robustness of Group I was tentative based on the lack of small sample-small effect nonsignificant result studies discovered in the funnel plot. Group II results (based upon 1,306 infants) are a stronger indication that, in general, infants with birth weights greater than 1,500 g are positively affected by the initiation of a stimulation program during hospitalization.     

Adult age differences in repetition priming: A meta-analysis.

This article reports a meta-analysis comparing the size of repetition priming effects in young and older adults. The main analysis included 39 effect sizes. Of these, 23 effect sizes could be classified as involving item priming and 16 as involving associative priming. The weighted mean effect size for the age difference was .304, with a confidence interval from .217 to .392. Because the confidence interval did not include 0, the hypothesis of no age difference in repetition priming was rejected. Subsidiary analyses, however, revealed that the weighted mean effect size for repetition priming was smaller than those for recognition or recall measures from the set of experiments included in the effect-size meta-analysis. Difficulties in drawing conclusions about process dissociations in old age in the face of only a partial dissociation in experimental measures are considered. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Forgetting the once-seen face: Estimating the strength of an eyewitness's memory representation.

The fidelity of an eyewitness's memory representation is an issue of paramount forensic concern. Psychological science has been unable to offer more than vague generalities concerning the relation of retention interval to memory trace strength for the once-seen face. A meta-analysis of 53 facial memory studies produced a highly reliable association (r=.18, d=0.37) between longer retention intervals and positive forgetting of once-seen faces, an effect equally strong for both face recognition and eyewitness identification studies. W. A. Wickelgren's (1974, 1975, 1977) theory of recognition memory provided statistically satisfactory fits to 11 different empirical forgetting functions. Applied to the results of field studies of eyewitness memory, the theory yields predictions relevant to fact finders' evaluations of eyewitness credibility. A plausible upper limit for witness initial memory strength corresponds to a probability of .67 of being correct on a fair six-person lineup. Furthermore, not only can the percentage of remaining memory strength be determined for any retention interval, but this strength estimate can be translated into an estimated probability of being correct on a fair lineup of a specified size. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

EEG characteristics prior to and following the evoked K-Complex.

Examined if the K-Complex reflects an arousal from sleep or a sleep protection mechanism. Ten 18–34-yr-old participants were presented auditory stimuli every 20 sec while asleep. Trials were sorted according to the presence or absence of a K-Complex. A fast Fourier Transformation of the data was computed on EEG segments prior to and following stimulus onset. The log power of activity in delta, theta, alpha, sigma, and beta bandwidths was computed. Results indicate that when a K-Complex was elicited, there were no differences in EEG activity prior to and following the stimulus. However, during slow wave sleep, when a K-Complex was not elicited, there was a significant overall increase in theta, alpha, sigma, and beta activity following stimulus. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effectiveness of grief therapy: A meta-analysis.

This meta-analysis addressed the question of how effective grief therapy is and for whom, using B. J. Becker's (1988) techniques for analyzing standardized mean-change scores. Analyses were based on 35 studies (N?=?2,284), with a weighted mean effect size (ES) of δ+?=?0.43 (95% confidence interval?=?0.33 to 0.52). Clients in no-treatment control groups showed little improvement (d=?=?0.06), possibly because of the relatively long delay between loss and treatment in most studies (mean delay?=?27 months). Moderators of treatment efficacy included time since loss and relationship to the deceased. Client selection procedures, a methodological factor not originally coded in this meta-analysis, appeared to contribute strongly to variability in ESs: a small number of studies involving self-selected clients produced relatively large ESs, whereas the majority of studies involving clients recruited by the investigators produced ESs in the small to moderate range. (PsycINFO Database Record (c) 2010 APA, all rights reserved)