Sampling variance in attenuated correlation coefficients: A Monte Carlo study.

Research in validity generalization has generated renewed interest in the sampling error of the Pearson correlation coefficient. The standard estimator for the sampling variance of the correlation was derived under assumptions that do not consider the presence of measurement error or range restriction in the data. The accuracy of the estimator in attenuated or restricted data has not been studied. This article presented the results of computer simulations that examined the accuracy of the sampling variance estimator in data containing measurement error. Sample sizes of n?=?25, n?=?60, and n?=?100 are used, with the reliability ranging from .10 to 1.00, and the population correlation ranging from .10 to 0.90. Results demonstrated that the estimator has a slight negative bias, but may be sufficiently accurate for practical applications if the sample size is at least 60. In samples of this size, the presence of measurement error does not add greatly to the inaccuracy of the estimator. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sampling variance in the correlation coefficient under indirect range restriction: Implications for validity generalization.

The authors conducted Monte Carlo simulations to investigate whether indirect range restriction (IRR) on 2 variables X and Y increases the sampling error variability in the correlation coefficient between them. The manipulated parameters were (a) IRR on X and Y (i.e., direct restriction on a third variable Z), (b) population correlations ρxy, ρxz, and ρyz and (c) sample size. IRR increased the sampling error variance in rxy to values as high as 8.50% larger than the analytically derived expected values. Thus, in the presence of IRR, validity generalization users need to make theory-based decisions to ascertain whether the effects of IRR are artifactual or caused by situational-specific moderating effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Monte Carlo study of three approaches to nonorthogonal analysis of variance.

Several alternative procedures have been advocated for analyzing nonorthogonal ANOVA data. Two in particular, J. E. Overall and D. K. Spiegel's (see record 1970-01534-001) Methods 1 and 2, have been the focus of controversy. A Monte Carlo study was undertaken to explore the relative sensitivity and error rates of these 2 methods, in addition to M. I. Applebaum and E. M. Cramer's (see record 1974-28956-001) procedure. Results of 2,250 3?×?3 ANOVAs conducted with each method and involving 3 underlying groups of population effects supported 3 hypotheses raised in the study: (a) Method 2 was more powerful than Method 1 in the absence of interaction; (b) Method 2 was biased upwards in the presence of interaction; and (c) Methods 1 and 2 both had Type I error rates close to those expected in the absence of interaction. In addition, it was found that in the absence of interaction, the Appelbaum and Cramer procedure was more powerful than Method 2 but slightly increased the Type I error rate. (16 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Monotonic hypotheses in multiple group designs: A Monte Carlo study.

Monotonic hypotheses are predictions about the ordering of group population means. A journal survey revealed that the problem was very common and that there was little uniformity among researchers regarding the statistical test to use. Most of the approaches in the literature to detect both monotonic trend and nonmonotonicity were compared under varying population conditions in a Monte Carlo simulation. The results suggested that only rarely will sample means order the same as the corresponding population means, leaving the approaches most researchers used with far too little power. Trend tests had far greater power; the one recommended is the familiar linear trend test. However, used alone this test does not detect the presence of any instances of nonmonotonicity. Therefore, it should be used in combination with a technique that can detect such inversions, preferably the Sidák-corrected reversal test conducted with a very high α (.50). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Component analysis versus common factor analysis: A Monte Carlo study.

Compares component and common factor analysis using 3 levels of population factor pattern loadings (.40, .60, .80) for each of the 3 levels of variables (9, 18, 36). Common factor analysis was significantly more accurate than components in reproducing the population pattern in each of the conditions examined. The differences decreased as the number of variables and the size of the population pattern loadings increased. The common factor analysis loadings were unbiased, had a smaller standard error than component loadings, and presented no boundary problems. Component loadings were significantly and systematically inflated even with 36 variables and loadings of .80. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods.

The efficacy of the Hedges and colleagues, Rosenthal-Rubin, and Hunter-Schmidt methods for combining correlation coefficients was tested for cases in which population effect sizes were both fixed and variable. After a brief tutorial on these meta-analytic methods, the author presents 2 Monte Carlo simulations that compare these methods for cases in which the number of studies in the meta-analysis and the average sample size of studies were varied. In the fixed case the methods produced comparable estimates of the average effect size; however, the Hunter-Schmidt method failed to control the Type I error rate for the associated significance tests. In the variable case, for both the Hedges and colleagues and Hunter-Schmidt methods, Type I error rates were not controlled for meta-analyses including 15 or fewer studies and the probability of detecting small effects was less than .3. Some practical recommendations are made about the use of meta-analysis. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A Monte Carlo evaluation of three formula estimates of cross-validated multiple correlation.

To determine the stability of regression equations, researchers have typically employed a cross-validation design in which weights are developed on an estimation subset of the sample and then applied to the members of a holdout sample. The present study used a Monte Carlo simulation to ascertain the accuracy with which the shrinkage in R–2 could be estimated by 3 formulas developed for this purpose. Results indicate that R. B. Darlington's (see record 1968-08053-001) and F. M. Lord (1950) and G. E. Nicholson's (1960) formulas yielded mean estimates approximately equal to actual cross-validation values, but with smaller standard errors. Although the Wherry estimate is a good estimate of population multiple correlation, it is an overestimate on population cross-validity. It is advised that the researcher estimate weights on the total sample to maximize the stability of the regression equation and then estimate the shrinkage in R–2 that he/she can expect when going to a new sample with either the Lord-Nicholson or Darlington estimation formulas. (17 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)