| Abstract: | A statistical test leads to a Type I error whenever it leads to the rejection of a null hypothesis that is in fact true. The probability of making a Type I error can be characterized in the following 3 ways: the conditional prior probability, the overall prior probability, and the conditional posterior probability. In this article, we show (a) that the alpha level can be equated with the 1st of these and (b) that it provides an upper bound for the second but (c) that it does not provide an estimate of the third, although it is commonly assumed to do so. We trace the source of this erroneous assumption first to statistical texts used by psychologists, which are generally ambiguous about which of the 3 interpretations is intended at any point in their discussions of Type I errors and which typically confound the conditional prior and posterior probabilities. Underlying this, however, is a more general fallacy in reasoning about probabilities, and we suggest that this may be the result of erroneous inferences about probabilistic conditional statements. Finally, we consider the possibility of estimating the (posterior) probability of a Type I error in situations in which the null hypothesis is rejected and, hence, the proportion of statistically significant results that may be Type I errors. (PsycINFO Database Record (c) 2010 APA, all rights reserved) |
