Confidence intervals for gamma-family measures of ordinal association.

[Correction Notice: An erratum for this article was reported in Vol 13(1) of Psychological Methods (see record 2008-02525-006). The note corrects simulation results presented in the article concerning the performance of confidence intervals (CIs) for Spearman's rs. An error in the author's C++ code affected all simulation results for Spearman's rs (but none of the results for gamma-family indices).] This research focused on confidence intervals (CIs) for 10 measures of monotonic association between ordinal variables. Standard errors (SEs) were also reviewed because more than 1 formula was available per index. For 5 indices, an element of the formula used to compute an SE is given that is apparently new. CIs computed with different SEs were compared in simulations with small samples (N = 25, 50, 75, or 100) for variables with 4 or 5 categories. With N > 25, many CIs performed well. Performance was best for consistent CIs due to N. Cliff and colleagues (N. Cliff, 1996; N. Cliff & V. Charlin, 1991; J. D. Long & N. Cliff, 1997). CIs for Spearman's rank correlation were also examined: Parameter coverage was erratic and sometimes egregiously underestimated. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Correction for Attenuation Due to Measurement Error: Clarifying Concepts and Creating Confidence Sets.

The correction for attenuation due to measurement error (CAME) has received many historical criticisms, most of which can be traced to the limited ability to use CAME inferentially. Past attempts to determine confidence intervals for CAME are summarized and their limitations discussed. The author suggests that inference requires confidence sets that demarcate those population parameters likely to have produced an obtained value-rather than indicating the samples likely to be produced by a given population-and that most researchers tend to confuse these 2 types of confidence sets. Three different Monte-Carlo methods are presented, each offering a different way of examining confidence sets under the new conceptualization. Exploring the implications of these approaches for CAME suggests potential consequences for other statistics. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Researchers Misunderstand Confidence Intervals and Standard Error Bars.

Little is known about researchers' understanding of confidence intervals (CIs) and standard error (SE) bars. Authors of journal articles in psychology, behavioral neuroscience, and medicine were invited to visit a Web site where they adjusted a figure until they judged 2 means, with error bars, to be just statistically significantly different (p     

Confidence intervals and replication: Where will the next mean fall?

Confidence intervals (CIs) give information about replication, but many researchers have misconceptions about this information. One problem is that the percentage of future replication means captured by a particular CI varies markedly, depending on where in relation to the population mean that CI falls. The authors investigated the distribution of this percentage for ? known and unknown, for various sample sizes, and for robust CIs. The distribution has strong negative skew: Most 95% CIs will capture around 90% or more of replication means, but some will capture a much lower proportion. On average, a 95% CI will include just 83.4% of future replication means. The authors present figures designed to assist understanding of what CIs say about replication, and they also extend the discussion to explain how p values give information about replication. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Bootstrap Standard Error and Confidence Intervals for the Correlation Corrected for Range Restriction: A Simulation Study.

The standard Pearson correlation coefficient is a biased estimator of the true population correlation, ρ, when the predictor and the criterion are range restricted. To correct the bias, the correlation corrected for range restriction, rc, has been recommended, and a standard formula based on asymptotic results for estimating its standard error is also available. In the present study, the bootstrap standard-error estimate is proposed as an alternative. Monte Carlo simulation studies involving both normal and nonnormal data were conducted to examine the empirical performance of the proposed procedure under different levels of ρ, selection ratio, sample size, and truncation types. Results indicated that, with normal data, the bootstrap standard-error estimate is more accurate than the traditional estimate, particularly with small sample size. With nonnormal data, performance of both estimates depends critically on the distribution type. Furthermore, the bootstrap bias-corrected and accelerated interval consistently provided the most accurate coverage probability for ρ. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Alpha's Standard Error (ASE): An Accurate and Precise Confidence Interval Estimate.

This research presents the inferential statistics for Cronbach's coefficient alpha on the basis of the standard statistical assumption of multivariate normality. The estimation of alpha's standard error (ASE) and confidence intervals are described, and the authors analytically and empirically investigate the effects of the components of these equations. The authors then demonstrate the superiority of this estimate compared with previous derivations of ASE in a separate Monte Carlo simulation. The authors also present a sampling error and test statistic for a test of independent sample alphas. They conclude with a recommendation that all alpha coefficients be reported in conjunction with standard error or confidence interval estimates and offer SAS and SPSS programming codes for easy implementation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confidence intervals for effect sizes: Compliance and clinical significance in the Journal of Consulting and Clinical Psychology..

Objective: In 2005, the Journal of Consulting and Clinical Psychology (JCCP) became the first American Psychological Association (APA) journal to require statistical measures of clinical significance, plus effect sizes (ESs) and associated confidence intervals (CIs), for primary outcomes (La Greca, 2005). As this represents the single largest editorial effort to improve statistical reporting practices in any APA journal in at least a decade, in this article we investigate the efficacy of that change. Method: All intervention studies published in JCCP in 2003, 2004, 2007, and 2008 were reviewed. Each article was coded for method of clinical significance, type of ES, and type of associated CI, broken down by statistical test (F, t, chi-square, r/R2, and multivariate modeling). Results: By 2008, clinical significance compliance was 75% (up from 31%), with 94% of studies reporting some measure of ES (reporting improved for individual statistical tests ranging from η2 = .05 to .17, with reasonable CIs). Reporting of CIs for ESs also improved, although only to 40%. Also, the vast majority of reported CIs used approximations, which become progressively less accurate for smaller sample sizes and larger ESs (cf. Algina & Kessleman, 2003). Conclusions: Changes are near asymptote for ESs and clinical significance, but CIs lag behind. As CIs for ESs are required for primary outcomes, we show how to compute CIs for the vast majority of ESs reported in JCCP, with an example of how to use CIs for ESs as a method to assess clinical significance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confidence intervals for the overall effect size in random-effects meta-analysis.

One of the main objectives in meta-analysis is to estimate the overall effect size by calculating a confidence interval (CI). The usual procedure consists of assuming a standard normal distribution and a sampling variance defined as the inverse of the sum of the estimated weights of the effect sizes. But this procedure does not take into account the uncertainty due to the fact that the heterogeneity variance (τ2) and the within-study variances have to be estimated, leading to CIs that are too narrow with the consequence that the actual coverage probability is smaller than the nominal confidence level. In this article, the performances of 3 alternatives to the standard CI procedure are examined under a random-effects model and 8 different τ2 estimators to estimate the weights: the t distribution CI, the weighted variance CI (with an improved variance), and the quantile approximation method (recently proposed). The results of a Monte Carlo simulation showed that the weighted variance CI outperformed the other methods regardless of the τ2 estimator, the value of τ2, the number of studies, and the sample size. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Statistical inference for a linear function of medians: Confidence intervals, hypothesis testing, and sample size requirements.

When the distribution of the response variable is skewed, the population median may be a more meaningful measure of centrality than the population mean, and when the population distribution of the response variable has heavy tails, the sample median may be a more efficient estimator of centrality than the sample mean. The authors propose a confidence interval for a general linear function of population medians. Linear functions have many important special cases including pairwise comparisons, main effects, interaction effects, simple main effects, curvature, and slope. The confidence interval can be used to test 2-sided directional hypotheses and finite interval hypotheses. Sample size formulas are given for both interval estimation and hypothesis testing problems. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confidence intervals in repeated-measures designs: The number of observations principle.

Since the publication of Loftus and Masson’s (1994) method for computing confidence intervals (CIs) in repeated-measures (RM) designs, there has been uncertainty about how to apply it to particular effects in complex factorial designs. Masson and Loftus (2003) proposed that RM CIs for factorial designs be based on number of observations rather than number of participants. However, determining the correct number of observations for a particular effect can be complicated, given the variety of effects occurring in factorial designs. In this paper the authors define a general “number of observations” principle, explain why it obtains, and provide step-by-step instructions for constructing CIs for various effect types. The authors illustrate these procedures with numerical examples. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Beyond the "Self in Self-Efficacy: Spouse Confidence Predicts Patient Survival Following Heart Failure.

Ratings of patient efficacy to manage illness, made by 191 congestive heart failure patients and their spouses, were examined as predictors of patients' survival over the next 4 years. When considered alone, both the patient's self-efficacy and the spouse's confidence ratings predicted survival, but only spouse confidence remained significant when both partners" efficacy ratings were included in the same Cox regression model. The overlapping prognostic significance of spouse confidence and a global, multicomponent measure of marital quality positioned the former as a proxy for the latter, reflecting a fundamentally social protective factor in patient survival. Successful adaptation to heart failure appears to involve more than the patient's personal agency, and psychosocial data from spouses can improve prediction of patient outcomes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A comparison of error in five sorting procedures for ordinal ranking.

The effect or sorting procedures on ranking error was investigated. Different groups of Ss ranked a series of 50 stimulus cards using 5 different sorting methods. Significant differences in ranking errors among the 5 methods were observed, with a "free" procedure showing less error than "structured" procedures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Temporal measures of vocalization: Some methodological considerations": Correction to Swingle.

Reports an error in "Temporal measures of vocalization: Some methodological considerations" by Paul G. Swingle (Journal of Personality and Social Psychology, 1984[Dec], Vol 47[6], 1263-1280). The copyright notice was inadvertently omitted. The notice that should have appeared on the first page of this article is provided in the erratum. (The following abstract of the original article appeared in record 1985-11098-001.) Five studies--with 164 university students, 56 military personnel, and 4 elderly persons--examined methodological issues associated with temporal measures of vocalization. The simple measures of phonation, silence, and interrupt and measures of silence relative to phonation were found to be sensitive to task and emotional factors and were stable across experience. A procedure for analyzing interviews is presented, and potential applications of the temporal measures are discussed. (46 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"On measures of explained variance in nonrecursive structural equation models": Correction.

Reports an error in the original article by P. M. Bentler and T. Raykov (Journal of Applied Psychology, 2000 [Feb], Vol 85[1], 125–131). On page 127, an equation was printed incorrectly; here, the equation is printed correctly. (The following abstract of this article originally appeared in record 2000-03754-013.) Whereas measures of explained variance in a regression and an equation of a recursive structural equation model can be simply summarized by a standard R2 measure, this is not possible in nonrecursive models in which there are reciprocal interdependencies among variables. This article provides a general approach to defining variance explained in latent dependent variables of nonrecursive linear structural equation models. A new method of its estimation, easily implemented in EQS or LISREL and available in EQS 6, is described and illustrated. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confidence in Individual and Group Decision Making: When "Two Heads" Are Worse Than One.

The authors investigated students' accuracy and confidence judgments for course-related material in college classrooms. Under conditions of group work and instructor feedback, students produced higher exam accuracy scores working in groups than alone but at a cost of increased confidence for groups' wrong answers. Groups' high confidence for wrong answers generated the case when "two heads are worse than one." Students participating in groups that arrived at wrong exam answers gave higher confidence when wrong and lower confidence when correct for repeated items on a final exam. "Two heads" groups when wrong had no adverse effect on students' accuracy for repeated exam items. An intervention of lecture and readings on confidence calibration, metamemory, and overconfidence did not improve the students' accuracy-confidence judgments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Asymptotically distribution-free (ADF) interval estimation of coefficient alpha": Correction.

Reports an error in "Asymptotically distribution-free (ADF) interval estimation of coefficient alpha" by Alberto Maydeu-Olivares, Donna L. Coffman and Wolfgang M. Hartmann (Psychological Methods, 2007[Jun], Vol 12[2], 157-176). The sentence describing Equation 1 is incorrect. The corrected sentence is presented in the erratum. (The following abstract of the original article appeared in record 2007-07830-003.) The point estimate of sample coefficient alpha may provide a misleading impression of the reliability of the test score. Because sample coefficient alpha is consistently biased downward, it is more likely to yield a misleading impression of poor reliability. The magnitude of the bias is greatest precisely when the variability of sample alpha is greatest (small population reliability and small sample size). Taking into account the variability of sample alpha with an interval estimator may lead to retaining reliable tests that would be otherwise rejected. Here, the authors performed simulation studies to investigate the behavior of asymptotically distribution-free (ADF) versus normal-theory interval estimators of coefficient alpha under varied conditions. Normal-theory intervals were found to be less accurate when item skewness >1 or excess kurtosis >1. For sample sizes over 100 observations, ADF intervals are preferable, regardless of item skewness and kurtosis. A formula for computing ADF confidence intervals for coefficient alpha for tests of any size is provided, along with its implementation as an SAS macro. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A score method of constructing asymmetric confidence intervals for the mean of a rating scale item.

This article presents a generalization of the Score method of constructing confidence intervals for the population proportion (E. B. Wilson, 1927) to the case of the population mean of a rating scale item. A simulation study was conducted to assess the properties of the Score confidence interval in relation to the traditional Wald (A. Wald, 1943) confidence interval under a variety of conditions, including sample size, number of response options, extremeness of the population mean, and kurtosis of the response distribution. The results of the simulation study indicated that the Score interval usually outperformed the Wald interval, suggesting that the Score interval is a viable method of constructing confidence intervals for the population mean of a rating scale item. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Confidence intervals for standardized linear contrasts of means.

Most psychology journals now require authors to report a sample value of effect size along with hypothesis testing results. The sample effect size value can be misleading because it contains sampling error. Authors often incorrectly interpret the sample effect size as if it were the population effect size. A simple solution to this problem is to report a confidence interval for the population value of the effect size. Standardized linear contrasts of means are useful measures of effect size in a wide variety of research applications. New confidence intervals for standardized linear contrasts of means are developed and may be applied to between-subjects designs, within-subjects designs, or mixed designs. The proposed confidence interval methods are easy to compute, do not require equal population variances, and perform better than the currently available methods when the population variances are not equal. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Perceptions of past research": Correction.

Reports an error in the original article by D. Bruce and H. P. Bahrick (American Psychologist, 1992, Vol 47[2], 319–328). On page 326, Table 4 contained a typographical error. The corrected table is presented. The following abstract of this article originally appeared in record 1992-25028-001). Publication records were determined for 17 research topics in learning, memory, and perception. Topics varied in the initial and median years of publications, longevity, and size and shape of the distribution of publications. A total of 237 research scholars indicated their knowledge, perceptions, and evaluations of the issues. Familiarity with a problem depended on its age and when one was trained: Earlier PhDs were more familiar with older issues, and later PhDs with more recent issues. Topics perceived as the demonstration of a phenomenon were judged as less important, and those perceived as the investigation of a cognitive process as more important. Problems viewed as resolved were accorded more significance, and those abandoned because of paradigm shifts or intractability as less significance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Accuracy in parameter estimation for targeted effects in structural equation modeling: Sample size planning for narrow confidence intervals.

In addition to evaluating a structural equation model (SEM) as a whole, often the model parameters are of interest and confidence intervals for those parameters are formed. Given a model with a good overall fit, it is entirely possible for the targeted effects of interest to have very wide confidence intervals, thus giving little information about the magnitude of the population targeted effects. With the goal of obtaining sufficiently narrow confidence intervals for the model parameters of interest, sample size planning methods for SEM are developed from the accuracy in parameter estimation approach. One method plans for the sample size so that the expected confidence interval width is sufficiently narrow. An extended procedure ensures that the obtained confidence interval will be no wider than desired, with some specified degree of assurance. A Monte Carlo simulation study was conducted that verified the effectiveness of the procedures in realistic situations. The methods developed have been implemented in the MBESS package in R so that they can be easily applied by researchers. (PsycINFO Database Record (c) 2011 APA, all rights reserved)