Dependent variable reliability and the power of significance tests.

Assuming that the linear models for classical test theory and ANOVA hold simultaneously for some dependent variable, it is shown that 2 contradictory statements concerning the relationship between reliability and statistical power are both correct. J. E. Overall and J. A. Woodward (see PA, Vol 53:8623, 57:7284) showed that when the reliability of a difference or change score is zero, the power of a statistical test of a hypothesis of no change can be at a maximum. J. L. Fleiss (see record 1977-07259-001) found the opposite result (i.e., that the power of a statistical test of no pre–post change is at a maximum when the reliability of the difference or gain scores is equal to one). The role of the reliability of the dependent variable in statistical evaluations of controlled experiments is examined. It is argued that the conditions that yield high reliability coefficients are not necessarily optimal for significance testing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)