| Abstract: | A series of Monte Carlo computer simulations was conducted to investigate (a) the likelihood that meta-analysis will detect true differences in effect sizes rather than attributing differences to methodological artifact and (b) the likelihood that meta-analysis will suggest the presence of moderator variables when in fact differences in effect sizes are due to methodological artifact. The simulations varied the magnitude of the true population differences between correlations, the number of studies included in the meta-analysis, and the average sample size. Simulations were run both correcting and not correcting for measurement error. The power of 3 indices—the Schmidt-Hunter ratio of expected to observed variance, the Callender-Osburn procedure, and a chi-square test—to detect true differences was investigated. Results show that small true differences were not detected regardless of sample size and the number of studies and that moderate true differences were not detected with small numbers of studies or small sample sizes. Hence, there is a need for caution in attributing observed variation across studies to artifact. (9 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved) |
