The appropriate use of null hypothesis testing.

The many criticisms of null hypothesis testing suggest when it is not useful and what it should not be used for. This article explores when and why its use is appropriate. Null hypothesis testing is insufficient when size of effect is important, but it is ideal for testing ordinal claims relating the order of conditions, which are common in psychology. Null hypothesis testing also is insufficient for determining beliefs, but it is ideal for demonstrating sufficient evidential strength to support an ordinal claim, with sufficient evidence being 1 criterion for a finding entering the corpus of legitimate findings in psychology. The line between sufficient and insufficient evidence is currently set at p     

One cheer for null hypothesis significance testing.

Null hypothesis testing as a tool in research is defended. Six examples are offered of situations in which, if all the researcher could do was "reject H? at α?=?.05" the scientific contribution would still be substantial. The examples are drawn from physics, cosmology, psychology, geophysics, career counseling and theology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On the one-mediator null hypothesis of salary equity.

Responds to comments by S. D. McLaughlin (1982) and R. M. McFatter (1982) and states that when the one-mediator model is acceptable, there is no reason to argue for bias on the basis of mean differences in salary between matched groups. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

In praise of value judgments in null hypothesis testing…and of "accepting" the null hypothesis.

Comments on the article by R. L. Hagen (see record 1997-02239-002) on null hypothesis testing. Hagen's article is constrained to the logic of deduction from an already formulated null hypothesis and the scientific conclusions that are validly drawn from one statistical decision or another. It is argued that subjective value judgment preceding the construction of the null hypothesis is an obscure precursor of the scientific logic of null hypothesis testing and that subsequent actions taken in professional practice as a result of failure to reject the null hypothesis constitutes its acceptance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

In criticism of the null hypothesis statistical test.

Comments on the article by R. L. Hagen (see record 1997-02239-002) in praise of the null hypothesis statistical test (NHST). NHST, is, in fact, a probabilistic imitation of modus tollens (or of the mathematical procedure of proof by contradiction). However, once the reasoning is made probabilistic, the inference is no longer valid. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

In praise of the null hypothesis statistical test.

Jacob Cohen (see record 1995-12080-001) raised a number of questions about the logic and information value of the null hypothesis statistical test (NHST). Specifically, he suggested that: (1) The NHST does not tell us what we want to know; (2) the null hypothesis is always false; and (3) the NHST lacks logical integrity. It is the author's view that although there may be good reasons to give up the NHST, these particular points made by Cohen are not among those reasons. When addressing these points, the author also attempts to demonstrate the elegance and usefulness of the NHST. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Determining false alarm rates in null hypothesis testing research.

Supports J. Cohen's (see record 1995-12080-001) point regarding null hypothesis testing yet provides a less extreme example than the one used by Cohen. The figure offered here gives researchers a way of more accurately gauging the importance of Cohen's point for their own research, and then determining what steps would be needed to obtain acceptable false alarm rates. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

How probable is the null hypothesis?

Comments on the J. Krueger (see record 2001-16601-002) discussion on null hypothesis significance testing. The current author extends Krueger's discussion of the debate over the probability of the null hypothesis, using Bayesian methods to resolve the debate. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

History of the sleeper effect: Some logical pitfalls in accepting the null hypothesis.

The history of research on the sleeper effect prior to 1978 can be divided into 5 stages: (a) initial discovery of the effect, (b) development of the underlying theory, (c) widespread acceptance of the effect and of the discounting cue explanation of it, (d) realization that past operational definitions of the effect were not isomorphic with the conceptual definition, and (e) repeated failure to demonstrate the effect once operational definitions were employed that corresponded to the conceptual definition (P. M. Gillig and A. G. Greenwald, 1974). These failures resulted in an invitation to accept the null hypothesis and to "lay sleeper effect to rest." This article illustrates why it is not justifiable to accept the null hypothesis about the sleeper effect. It is suggested that provisional acceptance of the null hypothesis depends on assuming that all the necessary theoretical, countervailing, statistical, and procedural conditions for an adequate test of the effect have been demonstrably met. It is further suggested that none of the empirical studies prior to 1978 demonstrably succeeded in meeting these conditions. However, adequate tests following the guidelines described by the authors for provisionally accepting the null hypothesis have recently been conducted, and the effect has been repeatedly found. A deductive model of the logical factors that should guide provisional acceptance of the null hypothesis is contrasted with a current model that stresses induction and statistical power analyses. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Methodological versus empirical literature: Two views on casual acceptance of the null hypothesis.

Presents examples of casual acceptance of the null hypothesis in respected journals. This occurs when authors are unaware of the problems discussed in the methodology literature. The literature proposes 4 criteria for a null conclusion, concerning problems with the data, probability, power of the research design, and persuasion based on overall relationships. The problem of inappropriate null conclusions should be solved by more attention in research to overall relationships and replications. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

If the null hypothesis is impossible, why test it?

Suggests that tests of the null hypothesis (NH) are informative only when the prior probability of the NH is reasonably large. In many situations, the NH represents a zero effect. An alternative approach suggests that researchers specify a good-enough belt that represents the range of outcomes that would be consistent with the hypothesis that a treatment had no effect. Focus is away from issues of statistical significance and toward the substantive significance of the results. Such a procedure also may result in NHs that have some probability of being true. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The question of cultural bias in assessment and diagnosis of ethnic minority clients: Let's reject the null hypothesis.

Cultural bias in the mental health assessment and psychiatric diagnosis of ethnic minorities has been asserted and challenged on the basis of clinical impression and a variety of psychometric criteria. The lack of a definitive empirical basis to resolve issues of assessment and diagnostic bias means that the null hypothesis (no bias or cross-cultural uniformity) prevails. This article argues that the traditional hypothesis to be nullified should be challenged. The consequences associated with its incorrect retention (Type II error?=?disservice to minority clients) may be more serious than its incorrect rejection (Type I error?=?misdirection to the mental health service system). If a client-centered error is judged more serious than a service system error, then a statement of bias or cross-cultural variance should be the null hypothesis until such time as empirical data suggest otherwise. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The gender reality hypothesis.

Comments on the article by J. S. Hyde (see record 2005-11115-001), in which Hyde reviewed meta-analytic evidence on gender differences and concluded that most psychological gender differences are in the close-to-zero or small range. The current author notes some omissions from Hyde's review, including the findings through other research large gender differences are reflected in some kinds of interests and occupational preferences, in males' and females placement on the people-things dimension of interests, and in many kinds of mental illness and behavior problems. The current author's position is that that many psychological gender differences are small-to-nonexistent, some are moderate, and some are large. The task that confronts gender researchers is to explain the complex profile of psychological gender differences and to untangle the myriad social and biological factors that generate both gender differences and gender similarities. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Testing the null hypothesis in meta-analysis: A comparison of combined probability and confidence interval procedures.

Combined significance tests (combined p values) and tests of the weighted mean effect size are used to combine information across studies in meta-analysis. A combined significance test (Stouffer test) is compared with a test based on the weighted mean effect size as tests of the same null hypothesis. The tests are compared analytically in the case in which the within-group variances are known and compared through large-sample theory in the more usual case in which the variances are unknown. Generalizations suggested are then explored through a simulation study. This work demonstrates that the test based on the average effect size is usually more powerful than the Stouffer test unless there is a substantial negative correlation between within-study sample size and effect size. Thus, the test based on the average effect size is generally preferable, and there is little reason to also calculate the Stouffer test. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The matching hypothesis reexamined.

Examined the assertion that men and women of similar attractiveness levels are drawn to one another as romantic partners in 3 computer-simulated experiments with 1,000 hypothetical couples. In the 1st mate selection simulation, the hypothetical Ss were given no awareness of their own attractiveness level but were programmed to demand an attractive partner; in the 2nd simulation, Ss sought a partner who matched their own awareness level; in the 3rd simulation, both of these criteria were used. Each simulation resulted in a significant intracouple attractiveness correlation. The simulation based on pure attractiveness-seeking produced a correlation in the upper range of those reported in actual studies of existing couples. It is argued that the use of models provided by simulations is a means of facilitating backward inference from system-wide patterns to the individual choices and behaviors that may produce these patterns. (38 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The proof of the pudding: An illustration of the relative strengths of null hypothesis, meta-analysis, and Bayesian analysis.

Some methodologists have recently suggested that scientific psychology's overreliance on null hypothesis significance testing (NHST) impedes the progress of the discipline. In response, a number of defenders have maintained that NHST continues to play a vital role in psychological research. Both sides of the argument to date have been presented abstractly. The authors take a different approach to this issue by illustrating the use of NHST along with 2 possible alternatives (meta-analysis as a primary data analysis strategy and Bayesian approaches) in a series of 3 studies. Comparing and contrasting the approaches on actual data brings out the strengths and weaknesses of each approach. The exercise demonstrates that the approaches are not mutually exclusive but instead can be used to complement one another. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The error of accepting the "theoretical" null hypothesis: The rise, fall, and resurrection of commonsense hypotheses in psychology.

When psychologists test a commonsense (CS) hypothesis and obtain no support, they tend to erroneously conclude that the CS belief is wrong. In many such cases it appears, after many years, that the CS hypothesis was valid after all. It is argued that this error of accepting the "theoretical" null hypothesis reflects confusion between the operationalized hypothesis and the theory or generalization that it is designed to test. That is, on the basis of reliable null data one can accept the operationalized null hypothesis (e.g., "A measure of attitude x is not correlated with a measure of behavior y"). In contrast, one cannot generalize from the findings and accept the abstract or theoretical null (e.g., "We know that attitudes do not predict behavior"). The practice of accepting the theoretical null hypothesis hampers research and reduces the trust of the public in psychological research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)