Power analysis and determination of sample size for covariance structure modeling.

A framework for hypothesis testing and power analysis in the assessment of fit of covariance structure models is presented. We emphasize the value of confidence intervals for fit indices, and we stress the relationship of confidence intervals to a framework for hypothesis testing. The approach allows for testing null hypotheses of not-good fit, reversing the role of the null hypothesis in conventional tests of model fit, so that a significant result provides strong support for good fit. The approach also allows for direct estimation of power, where effect size is defined in terms of a null and alternative value of the root-mean-square error of approximation fit index proposed by J. H. Steiger and J. M. Lind (1980). It is also feasible to determine minimum sample size required to achieve a given level of power for any test of fit in this framework. Computer programs and examples are provided for power analyses and calculation of minimum sample sizes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A statistics primer. Power analysis and sample size determination

Gynecologic-oncologic patients are at increased risk for complications with closed laparoscopy. Open laparoscopy eliminates the steps of blind insufflation and trocar insertion. This study is the first large series of open laparoscopies to assess the feasibility and safety of the open laparoscopy technique in patients with gynecologic malignancies. We performed 90 open laparoscopies in 89 oncologic patients with previous major surgery (65%) and/or radiotherapy (17%) or a large omental cake (18%). Complications due to the laparoscopic access technique occurred in one patient (1%) for whom a laparotomy was performed for a small bowel perforation. The incidence of complications of the open laparoscopy technique (1%) is favorable compared to the complication rate of closed laparoscopy in gynecologic-oncologic patients. It is concluded that open laparoscopy is a safe and feasible technique in gynecologic-oncologic patients.     

Power and sample size calculations for studies involving linear regression

This article presents methods for sample size and power calculations for studies involving linear regression. These approaches are applicable to clinical trials designed to detect a regression slope of a given magnitude or to studies that test whether the slopes or intercepts of two independent regression lines differ by a given amount. The investigator may either specify the values of the independent (x) variable(s) of the regression line(s) or determine them observationally when the study is performed. In the latter case, the investigator must estimate the standard deviation(s) of the independent variable(s). This study gives examples using this method for both experimental and observational study designs. Cohen's method of power calculations for multiple linear regression models is also discussed and contrasted with the methods of this study. We have posted a computer program to perform these and other sample size calculations on the Internet (see http://www.mc.vanderbilt.edu/prevmed/psintro+ ++.htm). This program can determine the sample size needed to detect a specified alternative hypothesis with the required power, the power with which a specific alternative hypothesis can be detected with a given sample size, or the specific alternative hypotheses that can be detected with a given power and sample size. Context-specific help messages available on request make the use of this software largely self-explanatory.     

Utility of intelligence tests for treatment planning, classification, and placement decisions: Recent empirical findings and future directions.

Intelligence testing in schools continues to be a frequent behavior among many school psychologists that creates a lucrative market for test publishers. Each year, between 1.5 and 1.8 million intelligence tests are administered by school psychologists. The typical school psychologist spends approximately two-thirds of his or her time in special education eligibility determination. This article maintains that intelligence tests contribute little if any information useful for the planning, implementation, and evaluation of instructional interventions for children. This argument is supported by the virtual absence of empirical evidence supporting the existence of aptitude?×?treatment interactions. We also suggest that intelligence tests are not particularly useful in making differential diagnostic and classification determinations for children with mild learning problems. Further, school study teams often ignore results of intelligence testing in making classification and placement decisions. We argue that intelligence testing is not a cost-beneficial use of school psychologists' time and should be replaced with assessment procedures directly related to the design, implementation, and evaluation of instructional interventions for children and youth. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Significance tests and the duplicity of binary decisions.

Presents a logical justification for the following statements and discusses their implications: It is duplicitous (misleading) to use significance tests for making binary (either/or) decisions regarding the validity of a theory; the binary choice between calling results significant or not significant should not govern the confidence placed in a theory, because such confidence cannot be gained in the either/or fashion characterizing deductive certainty. The implications include grounds for describing ways that effect size estimates become useful in making judgments about the value of theories. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multiple correlation: Exact power and sample size calculations.

This article discusses power and sample size calculations for observational studies in which the values of the independent variables cannot be fixed in advance but are themselves outcomes of the study. It reviews the mathematical framework applicable when a multivariate normal distribution can be assumed and describes a method for calculating exact power and sample sizes using a series expansion for the distribution of the multiple correlation coefficient. A table of exact sample sizes for level .05 tests is provided. Approximations to the exact power are discussed, most notably those of J. Cohen (1977). A rigorous justification of Cohen's approximations is given. Comparisons with exact answers show that the approximations are quite accurate in many situations of practical interest. More extensive tables and a computer program for exact calculations can be obtained from the authors. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Type I error rates for moderated multiple regression analysis.

The method of moderated multiple regression is increasingly being applied in the search for moderator variables in industrial and organizational psychology. Because of frequent failures of the method in revealing moderator effects in empirical studies—in which such effects are strongly expected—it has been suggested that the procedure may lack statistical power with respect to hypothesis tests about moderating effects and, therefore, is inappropriate for the purposes of conventional moderator analyses. We evaluated this conclusion with computer simulation data. Our study indicated that the method is not overly conservative and that the Type I error rate of moderated multiple regression is approximately .05 at α?=?.05. Moreover, a proposed alternative multivariate procedure, principal component regression, is shown to have a Type I error rate that approaches unity under ordinary conditions when applied to the evaluation of moderator effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Power, affiliation, and war: Three tests of a motivational model.

From previous research, a model relating war outbreak to a pattern of high power motivation and low affiliation motivation is developed and tested by content analysis of (1) historical materials from over 300 yrs of British history, (2) British–German communications at the outbreak of World War I, and (3) US–Soviet communications during the Cuban Missile Crisis. In all 3 studies, observed motive patterns in relation to war-vs-peace outcomes support the essential features of the model. Furthermore, wars once underway end only after power motivation drops and not after affiliation increases. Finally, ending of wars is followed by declines in power and achievement motivation. Alternative interpretations of the results are discussed and further, multivariate research is outlined. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Statistical inference for a linear function of medians: Confidence intervals, hypothesis testing, and sample size requirements.

When the distribution of the response variable is skewed, the population median may be a more meaningful measure of centrality than the population mean, and when the population distribution of the response variable has heavy tails, the sample median may be a more efficient estimator of centrality than the sample mean. The authors propose a confidence interval for a general linear function of population medians. Linear functions have many important special cases including pairwise comparisons, main effects, interaction effects, simple main effects, curvature, and slope. The confidence interval can be used to test 2-sided directional hypotheses and finite interval hypotheses. Sample size formulas are given for both interval estimation and hypothesis testing problems. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing for adverse impact when sample size is small.

Adverse impact evaluations often call for evidence that the disparity between groups in selection rates is statistically significant, and practitioners must choose which test statistic to apply in this situation. To identify the most effective testing procedure, the authors compared several alternate test statistics in terms of Type I error rates and power, focusing on situations with small samples. Significance testing was found to be of limited value because of low power for all tests. Among the alternate test statistics, the widely-used Z-test on the difference between two proportions performed reasonably well, except when sample size was extremely small. A test suggested by G. J. G. Upton (1982) provided slightly better control of Type I error under some conditions but generally produced results similar to the Z-test. Use of the Fisher Exact Test and Yates's continuity-corrected chi-square test are not recommended because of overly conservative Type I error rates and substantially lower power than the Z-test. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The experiment as the unit for computing rates of error.

In reply to Wilson (see 37: 5919), arguments are presented for controlling error rates per experiment rather than per hypothesis. The major contention is that a per hypothesis strategy permits wide variation in the total amount of error expected for different experimental designs involving the same total number of observations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The bigger the better: Productivity, size, and turnover in a sample of psychology departments.

Investigated whether the "older psychology departments in certain private universities were more productive than the psychology departments in a sample of other universities when department size was controlled." Number of publications and department turnover, relating to its size, age, and control were studied in 28 American psychology departments. Results suggested that (1) prior to 1960 older departments were more productive than newer departments, and (2) in the 1960s, departments were characterized by significantly different turnover rates. (27 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Conjunction error rates on a continuous recognition memory test: Little evidence for recollection.

Two experiments examined conjunction memory errors on a continuous recognition task where the lag between parent words (e.g., blackmail, jailbird) and later conjunction lures (blackbird) was manipulated. In Experiment 1, contrary to expectations, the conjunction error rate was highest at the shortest lag (1 word) and decreased as the lag increased. In Experiment 2 the conjunction error rate increased significantly from a 0- to a 1-word lag, then decreased slightly from a 1- to a 5-word lag. The results provide mixed support for simple familiarity and dual-process accounts of recognition. Paradoxically, searching for an item in memory does not appear to be a good encoding task. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of size of sample on eigenvalues, observed communalities, and factor loadings.

Several samples were randomly selected from a finite population of undergraduates over 5 N levels, factor analyzed, and then compared to the factor analysis of the finite population. Results indicate that within the limitations of the sampling design (Ns = 1,600, 400, 100, 25, and 17), samples representing N = 400 of a population of N = 2,322 yield factor structures comparable to that of the population. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relationship between chi-square and size of sample: the general case.

A table is given showing the size of Σf2obs required for significance at the .05 and .01 levels for from 20 to 100 observations in steps of 10, for chi-square analyses including from 3 to 10 cells. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Bootstrap Type I error rates for the correlation coefficient: An examination of alternate procedures.

This study examines three bootstrap procedures for estimating confidence intervals for the correlation coefficient. Results indicate that all three procedures produce slight inflation of the Type I error rate but that the inflation varies by method. Accordingly, choice of bootstrap procedure is an important consideration when testing hypotheses about correlations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Applications of standard error estimates in unrestricted factor analysis: Significance tests for factor loadings and correlations.

Estimates of standard errors of factor loadings and factor correlations in the unrestricted factor analysis model can be computed for oblique or orthogonal solutions under maximum likelihood. This information can be used to test individual coefficients for significance, to evaluate whether an orthogonal or oblique structure is most consistent with sample data, or to compute confidence intervals for single parameters or confidence regions for arbitrary groups of coefficients. Because the number of parameters estimated in factor analysis is approximately the product of number of variables multiplied by number of factors, a Bonferroni correction for the critical point of the individual test statistics is recommended to control the probability of a Type I error. Several examples are presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using Cohen's tables to determine the maximum power attainable in two-sample tests when one sample is limited in size.

Advantages of using equal sample sizes in 2 sample tests for means, correlations, and proportions are well known. However, in applied research there are frequently circumstances that limit the size of 1 of the 2 samples. This article draws attention to a simple method of determining, from J. Cohen's (1988) tables, the effects that this constraint (on the size of 1 sample) has on the maximum attainable power of these tests. These effects can be extremely serious in the case of small, medium, and even large effect sizes and clearly indicate that availability of a very large 2nd sample may not compensate for a constraint on the size of the 1st sample. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Unit-level voluntary turnover rates and customer service quality: Implications of group cohesiveness, newcomer concentration, and size.

Despite substantial growth in the service industry and emerging work on turnover consequences, little research examines how unit-level turnover rates affect essential customer-related outcomes. The authors propose an operational disruption framework to explain why voluntary turnover impairs customers’ service quality perceptions. On the basis of a sample of 75 work units and data from 5,631 employee surveys, 59,602 customer surveys, and organizational records, results indicate that unit-level voluntary turnover rates are negatively related to service quality perceptions. The authors also examine potential boundary conditions related to the disruption framework. Of 3 moderators studied (group cohesiveness, group size, and newcomer concentration), results show that turnover’s negative effects on service quality are more pronounced in larger units and in those with a greater concentration of newcomers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)