Increasing statistical power without increasing sample size.

Comments on the article by L. Wilkinson & the Task Force on Statistical Inference (TFSI) (see record 1999-03403-008) which was meant to initiate discussion in the field about changes in current practices of data analysis and reporting. The report was concerned with the use of statistical methods only and is not meant as an assessment of research methods in general. In this article, G. H. McClelland suggests that the TFSI may have created an unintended false impression that increasing sample size in the best or perhaps only remedy for improving statistical power. He notes that psychologists must consider design strategies other than augmenting sample size for increasing statistical power to reduce cost and minimize the burden on human or animal Ss. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Power and sample size calculations for studies involving linear regression

This article presents methods for sample size and power calculations for studies involving linear regression. These approaches are applicable to clinical trials designed to detect a regression slope of a given magnitude or to studies that test whether the slopes or intercepts of two independent regression lines differ by a given amount. The investigator may either specify the values of the independent (x) variable(s) of the regression line(s) or determine them observationally when the study is performed. In the latter case, the investigator must estimate the standard deviation(s) of the independent variable(s). This study gives examples using this method for both experimental and observational study designs. Cohen's method of power calculations for multiple linear regression models is also discussed and contrasted with the methods of this study. We have posted a computer program to perform these and other sample size calculations on the Internet (see http://www.mc.vanderbilt.edu/prevmed/psintro+ ++.htm). This program can determine the sample size needed to detect a specified alternative hypothesis with the required power, the power with which a specific alternative hypothesis can be detected with a given sample size, or the specific alternative hypotheses that can be detected with a given power and sample size. Context-specific help messages available on request make the use of this software largely self-explanatory.     

The auto-verification of social hypotheses: Stereotyping and the power of sample size.

In the literature on social hypothesis testing, the co-occurrence of 2 principles is often held responsible for hypothesis confirmation. The first is positive testing (e.g., looking for covert rather than overt aggression when testing the stereotype that female aggression is covert), and the second is a cooperative social environment that will often acquiesce and provide positive answers (i.e., positive examples for covert female aggression). However, it is argued that the co-occurrence of 1-sided questions and confirming answers does not logically verify a hypothesis. A theoretical framework is presented that explains why a constant ratio of confirming to disconfirming evidence has more impact when based on a large than on a small sample of observations. In 2 experiments, a constant affirmation rate led to auto-verification of the hypothesis that was represented by the larger sample. The enhanced significance of large samples is shown to be independent of stereotypical expectancies and unconfounded with diagnosticity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Internal bremsstrahlung: Exact relativistic independent-particle-approximation calculations

Recently, natural and social scientists have pointed out that the need to make scientific results apply to both sexes is not met by simply adding women as research subjects. They suggest that the social and biological specificity of both sexes must be recognized and adjustments made to the ways questions are asked, hypotheses are generated, research subjects are chosen, and data are analysed. It is important to examine the definitions, concepts and methods used in research in occupational health to see whether they obstruct recognition of women's occupational health problems or interfere with gender-neutral standard setting. Unravelling the effects of sex on occupational health is complicated by the fact that women and men do not, by and large, work at the same jobs. Definitions of "work" must in some cases be adjusted to take account of women's occupations, just as definitions involving "health" must include women's biological specificity. Appropriate changes must be made to generate sex-inclusive definitions of exposures to occupational hazards and of health effects. Methods for evaluating exposures typical of women's work must be developed. Women and their work must be appropriately included when standards for occupational exposures are set. If these adjustments are not made, women's occupational health problems will be seriously underestimated and understanding of health problems of both sexes will be hindered. Sociological analysis should also be done to reveal the mechanisms by which biased concepts and procedures develop and are reinforced.     

Exact equivalence test for risk ratio and its sample size determination under inverse sampling

When data are dichotomous, this paper notes the utility of inverse sampling in establishing equivalence with respect to the risk ratio. This paper develops an exact equivalence test that accounts for the risk ratio under inverse sampling and further discusses the relationship between the exact equivalence test and the exact conditional confidence limits. Also included are an exact and two asymptotic procedures for calculation of the minimum required number of index subjects for a desired power 1--beta at a given alpha-level. Finally, this paper provides a table that summarizes the minimum required number of index subjects for powers equal to 0.90 and 0.80 in application of the proposed exact equivalence test at 0.05-level in a variety of situations.     

Application of GEE procedures for sample size calculations in repeated measures experiments

Derivation of the minimum sample size is an important consideration in an applied research effort. When the outcome is measured at a single time point, sample size procedures are well known and widely applied. The corresponding situation for longitudinal designs, however, is less well developed. In this paper, we adapt the generalized estimating equation (GEE) approach of Liang and Zeger to sample size calculations for discrete and continuous outcome variables. The non-central version of the Wald Chi 2 test is considered. We use the damped exponential family of correlation structures described in Mu?oz et al. for the 'working' correlation matrix among the repeated measures. We present a table of minimum sample sizes for binary outcomes, and discuss extensions that account for unequal allocation, staggered entry and loss to follow-up.     

Multiple determinants and effect size.

Used a Monte Carlo study to investigate the magnitude of various relations among behaviors and traits in the context of a multiple-determinant framework (MDF). It was found that when only 3 traits determined each of 2 behaviors and the 2 behaviors were influenced by only 1 common trait, there was an upper bound correlation of about .30 between the 2 behaviors; there was also an upper bound correlation of about .50 between measures of the common trait and the behaviors. When only 4 traits determined each of the 2 behaviors with both behaviors being influenced by 1 common trait, there was an upper bound correlation of about .25 between the 2 behaviors and an upper bound correlation of about .45 between measures of the common trait and the behaviors. It is argued that researchers should abandon the implicit assumption of a one-to-one relation between traits and behavioral consequences and instead adopt an MDF in the study of behavior, focusing not only on the additive effects of multiple determinants but also on the interactions between these determinants. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Averaging correlation coefficients: Influence of heterogeneity and set size.

Silver and Dunlap (1987) recently investigated two methods for averaging correlations: (a) simple average of rs, and (b) average of Fisher's z transformations of r backtransformed to r. The latter method was found to be less positively biased than the simple average of rs was negatively biased. The current study extends these results by demonstrating that the z-transformation method is less susceptible to heterogeneity in the correlations that are averaged, but that the absolute magnitude of bias in the two approaches becomes similar as set size increases. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multiple regression and correlation techniques: Recent controversies and best practices.

Objective: This article presumes familiarity with the basics of multiple regression and correlation (MRC) methods and addresses recent controversies and emerging innovations. Areas of emphasis include linking analyses to theory-driven hypotheses, treatment of covariates in hierarchical regression models, recent debates about the testing of mediator and moderator hypotheses, and incorporating confidence intervals into reports of findings using MRC. Conclusions: Two important conceptual innovations (linking analyses closely to theory-derived hypotheses; focusing interpretations on effect sizes and confidence intervals rather than p values) can increase the scientific yield for researchers making use of MRC methods in rehabilitation psychology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relationship between chi-square and size of sample: the general case.

A table is given showing the size of Σf2obs required for significance at the .05 and .01 levels for from 20 to 100 observations in steps of 10, for chi-square analyses including from 3 to 10 cells. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A statistics primer. Power analysis and sample size determination

Gynecologic-oncologic patients are at increased risk for complications with closed laparoscopy. Open laparoscopy eliminates the steps of blind insufflation and trocar insertion. This study is the first large series of open laparoscopies to assess the feasibility and safety of the open laparoscopy technique in patients with gynecologic malignancies. We performed 90 open laparoscopies in 89 oncologic patients with previous major surgery (65%) and/or radiotherapy (17%) or a large omental cake (18%). Complications due to the laparoscopic access technique occurred in one patient (1%) for whom a laparotomy was performed for a small bowel perforation. The incidence of complications of the open laparoscopy technique (1%) is favorable compared to the complication rate of closed laparoscopy in gynecologic-oncologic patients. It is concluded that open laparoscopy is a safe and feasible technique in gynecologic-oncologic patients.     

Sample size and power

This paper sets forth the basic concepts of the calculation of sample size and power in clinical research. It provides the reader with a basic understanding of the relationship between sample size and power and the components within the research, such as the variability of the measure being used as the primary outcome. The paper also discusses a number of general issues related to sample size and power, such as the importance of the difference between clinical and statistical significance, how one approaches trials attempting to establish the equivalence of clinical interventions, and the critical need for appropriate consultation.     

Through a narrow window: Sample size and the perception of correlation.

A theoretical analysis (Y. Kareev, 1995b) of the sampling distribution of correlations led to the surprising conclusion that the use of small samples has a potential advantage for the early detection of a correlation. This is so because the distribution is highly skewed, and the smaller the sample size, the more the distribution is skewed. This article describes 2 experiments that were designed as empirical tests of this conclusion. In Experiment 1 (N?=?112), the authors compared the predictions of participants differing in their working-memory capacity (hence in the size of the samples they were likely to consider). In Experiment 2 (N?=?144), the authors compared the predictions of participants who viewed samples of different sizes, whose size was determined by the authors. The results fully supported Y. Kareev's conclusion: In both experiments, participants with lower capacity (or smaller samples) indeed perceived the correlation as more extreme and were more accurate in their predictions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size.

In confirmatory factor analysis, hypothesized models reflect approximations to reality so that any model can be rejected if the sample size is large enough. In the present article, we examine the influence of sample size on different fit indexes for both real and simulated data. Contrary to claims by Bentler and Bonett (1980), their incremental fit index was substantially affected by sample size. Contrary to claims by Joreskog and Sorbom (1981), their goodness-of-fit indexes provided by {lisrel} were substantially affected by sample size. Contrary to claims by Bollen (1986), his new incremental fit index was substantially affected by sample size. Hoelter's (1983) critical N index was also substantially affected by sample size. Of the more than 30 indexes considered, the Tucker-Lewis (1973) index was the only widely used index that was relatively independent of sample size. However, four new indexes based on the same form as the Tucker-Lewis index were also relatively independent of sample size. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Directional decisions for two-tailed tests: Power, error rates, and sample size.

One imposing directional decisions on nondirectional tests will overestimate power, underestimate sample size, and ignore the risk of Type III error (getting the direction wrong) if traditional calculations—those applying to nondirectional decisions—are used. Usually trivial with the z test, the errors might be important where α is large and effect size is small or with tests using other distributions. One can avoid the errors by using calculations that apply to directional decisions or by using a directional two-tailed test at the outset, a conceptually simpler solution. With a revised concept of power, this article shows calculations for the test; explains how to find its power, Type III error risk, and sample size in statistical tables for traditional tests; compares it to conventional one- and two-tailed tests and to one- and two-sided confidence intervals; and concludes that when a significance test is planned it is the best choice for most purposes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multiple dimensions of affective and cognitive disturbance: Latent-variable models in a community sample.

A clear understanding and confirmation of the structure of psychological distress has been hampered by different theoretical perspectives, ranges of measures, and methodologies. This study examined the latent construct structure of psychological distress as reflected in 27 self-report measures of psychological functioning from a community sample of 614 young adults. Models that hypothesized a single dimension of distress or 2 first-order factors (positive and negative affect) poorly fit the data. A model that fit well contained 9 first-order latent constructs, including Emotional Distress, Self-Derogation, Purpose in Life, Psychoticism, Depression, Anxiety, Hostility, Suicide Ideation, and Disorganized Thinking. Second and third higher order models achieved greater parsimony but were statistically inferior to the 9-factor model. Models are contrasted and discussed in terms of prior theory and current conceptualizations of mental health. (PsycINFO Database Record (c) 2010 APA, all rights reserved)