An effect size primer: A guide for clinicians and researchers.

Increasing emphasis has been placed on the use of effect size reporting in the analysis of social science data. Nonetheless, the use of effect size reporting remains inconsistent, and interpretation of effect size estimates continues to be confused. Researchers are presented with numerous effect sizes estimate options, not all of which are appropriate for every research question. Clinicians also may have little guidance in the interpretation of effect sizes relevant for clinical practice. The current article provides a primer of effect size estimates for the social sciences. Common effect sizes estimates, their use, and interpretations are presented as a guide for researchers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Statistical and Practical Significance in the IJPT: A Research Review from 1993-2003.

The reporting and interpretation of effect sizes in addition to statistical significance tests is becoming increasingly recognized as good research practice, as evidenced by the editorial policies of at least 23 journals that now require effect sizes. Statistical significance tests are limited in the information they provide readers about results, and effect sizes can be useful when evaluating result importance. The current article (a) summarizes statistical versus practical significance, (b) briefly discusses various effect size options, (c) presents a review of research articles published in the International Journal of Play Therapy (1993-2003) regarding use of effect sizes and statistical significance tests, and (d) provides recommendations for improved research practice in the journal and elsewhere. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Power and Measures of Effect Size in Analysis of Variance With Fixed Versus Random Nested Factors.

Ignoring a nested factor can influence the validity of statistical decisions about treatment effectiveness. Previous discussions have centered on consequences of ignoring nested factors versus treating them as random factors on Type I errors and measures of effect size (B. E. Wampold & R. C. Serlin, see record 2000-16737-003). The authors (a) discuss circumstances under which the treatment of nested provider effects as fixed as opposed to random is appropriate; (b) present 2 formulas for the correct estimation of effect sizes when nested factors are fixed; (c) present the results of Monte Carlo simulations of the consequences of treating providers as fixed versus random on effect size estimates, Type I error rates, and power; and (d) discuss implications of mistaken considerations of provider effects for the study of differential treatment effects in psychotherapy research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Reporting results of latent growth modeling and multilevel modeling analyses: Some recommendations for rehabilitation psychology.

Objective: There has been a general increase in interest and use of modeling techniques that treat data as nested, whether it is people nested within larger units, such as families or treatment centers, or observations nested under people. The popularity can be witnessed by noting the number of new textbooks and articles related to latent growth curve modeling and multilevel modeling. This paper discusses both of these techniques in the context of longitudinal research designs, with the main purposes of highlighting some benefits and issues related to the use of these models and outlining guidelines for reporting results from studies using multilevel modeling or latent growth modeling. Implications: These longitudinal analytic techniques can be greatly beneficial to researchers conducting rehabilitation studies, but there are several issues related to their use and reporting that need to be taken into consideration. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Taxometrics in Psychopathology Research: An Introduction to Some of the Procedures and Related Methodological Issues.

This article introduces a special section on the use of taxometrics to examine the categorical versus the dimensional structure of various forms of psychopathology. Paving the way into the special section, this introduction briefly describes 3 taxometric methods--mean above minus below a sliding cut (MAMBAC), maximum covariation (MAXCOV), and maximum eigenvalue (MAXEIG)--and discusses possible threats to statistical conclusion validity that often emerge when such techniques are applied in psychopathology research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Accurate tests of statistical significance for rWG and average deviation interrater agreement indexes.

The authors demonstrated that the most common statistical significance test used with rWG-type interrater agreement indexes in applied psychology, based on the chi-square distribution, is flawed and inaccurate. The chi-square test is shown to be extremely conservative even for modest, standard significance levels (e.g., .05). The authors present an alternative statistical significance test, based on Monte Carlo procedures, that produces the equivalent of an approximate randomization test for the null hypothesis that the actual distribution of responding is rectangular and demonstrate its superiority to the chi-square test. Finally, the authors provide tables of critical values and offer downloadable software to implement the approximate randomization test for rWG type and for average deviation (AD)-type interrater agreement indexes. The implications of these results for studying a broad range of interrater agreement problems in applied psychology are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies.

Underpowered studies persist in the psychological literature. This article examines reasons for their persistence and the effects on efforts to create a cumulative science. The "curse of multiplicities" plays a central role in the presentation. Most psychologists realize that testing multiple hypotheses in a single study affects the Type I error rate, but corresponding implications for power have largely been ignored. The presence of multiple hypothesis tests leads to 3 different conceptualizations of power. Implications of these 3 conceptualizations are discussed from the perspective of the individual researcher and from the perspective of developing a coherent literature. Supplementing significance tests with effect size measures and confidence intervals is shown to address some but not necessarily all problems associated with multiple testing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

From Theory to Application and Back Again: Implications of Research on Medical Expertise for Psychological Theory.

Research directed at an understanding of medical expertise is about 30 years old, and many developments in this literature parallel progress in cognitive psychology. Over the past 15 years or so, this research became much more closely identified with particular psychological theories. Initial forays into medicine were essentially direct applications of methods developed in the psychology lab to the more natural domain of medicine, with varying degrees of success. These attempts were followed by a second wave that took the psychological theories themselves more seriously in a more thoughtful application of psychological methods to the medical domain. I will argue in the present paper that the methods and theories used in the study of medical expertise have advanced to the point that there is some reverse flow and they are providing a unique and valuable perspective on the nature of thinking. (PsycINFO Database Record (c) 2010 APA, all rights reserved)