Intraclass correlation estimates in a school-based smoking prevention study. Outcome and mediating variables, by sex and ethnicity

Most school-based smoking prevention studies employ designs in which schools or classrooms are assigned to different treatment conditions while observations are made on individual students. This design requires that the treatment effect be assessed against the between-school variance. However, the between-school variance is usually larger than the variance that would be obtained if students were individually randomized to different conditions. Consequently, the power of the test for a treatment effect is reduced, and it becomes difficult to detect important treatment effects. To assess the potential loss of power or to calculate appropriate sample sizes, investigators need good estimates of the intraclass correlations for the variables of interest. The authors calculated intraclass correlations for some common outcome variables in a school-based smoking prevention study, using a three-level model-i.e., students nested within classrooms and classrooms nested within schools. The authors present the intraclass correlation estimates for the entire data set, as well as separately by sex and ethnicity. They also illustrate the use of these estimates in the planning of future studies.     

On the sampling variance of intraclass correlations and genetic correlations

Widely used standard expressions for the sampling variance of intraclass correlations and genetic correlation coefficients were reviewed for small and large sample sizes. For the sampling variance of the intraclass correlation, it was shown by simulation that the commonly used expression, derived using a first-order Taylor series performs better than alternative expressions found in the literature, when the between-sire degrees of freedom were small. The expressions for the sampling variance of the genetic correlation are significantly biased for small sample sizes, in particular when the population values, or their estimates, are close to zero. It was shown, both analytically and by simulation, that this is because the estimate of the sampling variance becomes very large in these cases due to very small values of the denominator of the expressions. It was concluded, therefore, that for small samples, estimates of the heritabilities and genetic correlations should not be used in the expressions for the sampling variance of the genetic correlation. It was shown analytically that in cases where the population values of the heritabilities are known, using the estimated heritabilities rather than their true values to estimate the genetic correlation results in a lower sampling variance for the genetic correlation. Therefore, for large samples, estimates of heritabilities, and not their true values, should be used.     

Planning for the appropriate analysis in school-based drug-use prevention studies.

School-based drug-use prevention studies often apply interventions to entire schools. A major problem for these studies results from the intragroup dependence often seen when intact social groups are assigned to study conditions. Analysis of data from 2 such studies revealed intraclass correlation coefficients between 0.02 and 0.05 for common drug use measures. Because even such modest intragroup dependence can invalidate the traditional fixed-effects analyses, researchers should adopt alternative methods that acknowledge this dependence. These alternative methods are reviewed, and appropriate methods for computing sample size requirements are illustrated. Investigators should consider these analysis issues when planning future studies, because the number of schools required for an unbiased analysis may be substantially greater than for the traditional methods. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interrater reliability and agreement of subjective judgments.

Indexes of interrater reliability and agreement are reviewed, and suggestions are made regarding their use in counseling psychology research. The distinction between agreement and reliability is clarified, and the relationships between these indexes and the level of measurement and type of replication are discussed. Indexes of interrater reliability appropriate for use with ordinal and interval scales are considered. The intraclass correlation as a measure of interrater reliability is discussed in terms of the treatment of between-raters variance and the appropriateness of reliability estimates based on composite or individual ratings. The advisability of optimal weighting schemes for calculating composite ratings is also considered. Measures of interrater agreement for ordinal and interval scales are described, as are measures of interrater agreement for data at the nominal level of measurement. (54 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

How interchangeable are different estimators of effect size?

The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analysis of prevention program effectiveness with clustered data using generalized estimating equations.

Experimental studies of prevention programs often randomize clusters of individuals rather than individuals to treatment conditions. When the correlation among individuals within clusters is not accounted for in statistical analysis, the standard errors are biased, potentially resulting in misleading conclusions about the significance of treatment effects. This study demonstrates the generalized estimating equations (GEE) method, focusing specifically on the GEE-independent method, to control for within-cluster correlation in regression models with either continuous or binary outcomes. The GEE-independent method yields consistent and robust variance estimates. Data from Project DARE, a youth substance abuse prevention program, are used for illustration. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On various intraclass correlation reliability coefficients.

Briefly reviews the most frequently used and misused intraclass correlation--analysis of variance (ANOVA) reliability measures. Recommendations are made for the 1-way ANOVA intraclass correlation and against the use of several coefficients: B. J. Winer's (1971) so-called "anchor point method," the Kuder-Richardson Formula Number 20, and the Spearman-Brown prediction formula. The computation of the intraclass correlation coefficient via 2-way ANOVA is not encouraged. Several uses and misuses of reliability coefficients applied to dichotomous data are illustrated. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs.

When a meta-analysis on results from experimental studies is conducted, differences in the study design must be taken into consideration. A method for combining results across independent-groups and repeated measures designs is described, and the conditions under which such an analysis is appropriate are discussed. Combining results across designs requires that (a) all effect sizes be transformed into a common metric, (b) effect sizes from each design estimate the same treatment effect, and (c) meta-analysis procedures use design-specific estimates of sampling variance to reflect the precision of the effect size estimates. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

"How interchangeable are different estimators of effect size?": Correction to Ray and Shadish (1996).

Reports an error in the original article by J. W. Ray and W. R. Shadish (Journal of Consulting and Clinical Psychology, 1996[Dec], Vol 64(6), 1316–1325). On page 1325, a correction is made to column 1, lines 25–26. (The following abstract of this article originally appeared in record 1996-07086-021). The computation of effect sizes is a key feature of meta-analysis. In treatment outcome meta-analyses, the standardized mean difference statistic on posttest scores (d) is usually the effect size statistic used. However, when primary studies do not report the statistics needed to compute d, many methods for estimating d from other data have been developed. Little is known about the accuracy of these estimates, yet meta-analysts frequently use them on the assumption that they are estimating the same population parameter as d. This study investigates that assumption empirically. On a sample of 140 psychosocial treatment or prevention studies from a variety of areas, the present study shows that these estimates yield results that are often not equivalent to d in either mean or variance. The frequent mixing of d and other estimates of d in past meta-analyses, therefore, may have led to biased effect size estimates and inaccurate significance tests. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of meta-analysis versus analysis of variance of individual patient data

Meta-analysis is a method of synthesizing the results of independent studies. We consider the case in which there are multiple treatments and a control, with the goal of estimating the relative effect of each treatment based on continuous outcomes. Even when all data are available, rather than only summary data, it has become common to use meta-analytic estimators of treatment contrasts. Alternatively, we could use a two-way analysis of variance model with no interaction in which one factor is study and one factor is treatment. For the unbalanced case, we obtain the surprising result that the standard meta-analysis estimates of treatment contrasts are identical to the least squares estimators of treatment contrasts in the linear model. Because a meta-analysis of individual patient data can be considerably more costly in terms of data retrieval than a meta-analysis of summary data, this equivalence provides for cost-efficient analysis.     

Issues for covariance analysis of dichotomous and ordered categorical data from randomized clinical trials and non-parametric strategies for addressing them

Analysis of covariance is an effective method for addressing two considerations for randomized clinical trials. One is reduction of variance for estimates of treatment effects and thereby the production of narrower confidence intervals and more powerful statistical tests. The other is the clarification of the magnitude of treatment effects through adjustment of corresponding estimates for any random imbalances between the treatment groups with respect to the covariables. The statistical basis of covariance analysis can be either non-parametric, with reliance only on the randomization in the study design, or parametric through a statistical model for a postulated sampling process. For non-parametric methods, there are no formal assumptions for how a response variable is related to the covariables, but strong correlation between response and covariables is necessary for variance reduction. Computations for these methods are straightforward through the application of weighted least squares to fit linear models to the differences between treatment groups for the means of the response variable and the covariables jointly with a specification that has null values for the differences that correspond to the covariables. Moreover, such analysis is similarly applicable to dichotomous indicators, ranks or integers for ordered categories, and continuous measurements. Since non-parametric covariance analysis can have many forms, the ones which are planned for a clinical trial need careful specification in its protocol. A limitation of non-parametric analysis is that it does not directly address the magnitude of treatment effects within subgroups based on the covariables or the homogeneity of such effects. For this purpose, a statistical model is needed. When the response criterion is dichotomous or has ordered categories, such a model may have a non-linear nature which determines how covariance adjustment modifies results for treatment effects. Insight concerning such modifications can be gained through their evaluation relative to non-parametric counterparts. Such evaluation usually indicates that alternative ways to compare treatments for a response criterion with adjustment for a set of covariables mutually support the same conclusion about the strength of treatment effects. This robustness is noteworthy since the alternative methods for covariance analysis have substantially different rationales and assumptions. Since findings can differ in important ways across alternative choices for covariables (as opposed to methods for covariance adjustment), the critical consideration for studies with covariance analyses planned as the primary method for comparing treatments is the specification of the covariables in the protocol (or in an amendment or formal plan prior to any unmasking of the study.     

Alcohol treatment changes in coping skills, self-efficacy, and levels of alcohol use and related problems 1 year following treatment initiation.

The relationships among alcohol treatment, coping skills, and self-efficacy in predicting alcohol use and related consequences following treatment initiation were investigated. The participants were 77 men and 65 women who were entering either inpatient or outpatient alcohol treatment. The analyses confirmed predictions that treatment, coping skills, and self-efficacy each contributed significantly to the prediction of 12-month alcohol consumption beyond the variance accounted for by participant control variables. Only self-efficacy explained significant additional variance in the consequences outcome. Mediation analyses of the alcohol consumption variables suggested that treatment effects were not mediated by either coping skills or self-efficacy and that the effects of coping skills were not mediated by self-efficacy. The findings are interpreted as providing partial support for social learning theory approaches. Suggestions for future research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Ultrastructural studies of spermiogenesis in rabbit exposed to chronic fluoride toxicity

OBJECTIVE: To compare skinfold thickness measurements with bioelectrical impedance analysis (BIA) as a measure of body fat for use in a survey of children (the National Study of Health and Growth). DESIGN: Part cross-sectional, part repeated measurement study. SETTING: A junior school in Bath. SUBJECTS: 42 boys and 33 girls aged from 9 to 11 years. INTERVENTIONS: Measurements of BIA, height, weight, and triceps, biceps, subscapular and suprailiac skinfolds. RESULTS: All measurements were highly repeatable with intraclass correlation coefficients > 0.90. The level of agreement between estimates of percentage of body fat derived from prediction equations based on impedance or skinfold measurements respectively was poor: the mean difference (impedance estimate minus skinfold estimate) was 4.67% (95% range -3.47 to 12.82) for boys and 7.81% (95% range 1.27 to 14.34) for girls. The two estimates were found to correlate highly (r = 0.83 for boys and r = 0.81 for girls) because weight, used to convert estimates of fat-free mass derived from impedance to fat mass, was highly correlated with impedance and moderately highly correlated with skinfold thicknesses. The correlations of resistance (R) and (H)2/R with skinfold thicknesses were very low. There was a moderate correlation of R and H2/R with log(weight-for-height index), but lower than that of log(weight-for-height index) with each of the skinfolds. CONCLUSIONS: As currently available equations for converting impedance-based estimates of total body water to fat mass are not fully developed for use in children of varying ages, estimates of body fat calculated from skinfold thickness measurements remain preferable in epidemiological studies of children's health and growth.     

Decision analysis for treatment of early stage prostate cancer

OBJECTIVE: The purpose of this study is to compare reports of alcohol use, drug use and sexual behavior from 30-day Summary measures with an expanded version of a Timeline Follow-back (Timeline) interview technique among gay/bisexual men entering outpatient substance abuse treatment at a gay-identified agency. METHOD: Respondents (N = 418) first completed self-administered questionnaires covering the 30-day period prior to their last use of alcohol or drugs. Summary measures included alcohol use, number of days of use for five categories of drugs and number of episodes of anal intercourse (with and without condoms) by partner type (primary or secondary). Participants then completed the Timeline interview procedure to recall their daily drinking, drug use and sexual behavior during the same 30-day period. RESULTS: The findings indicate that the Timeline method yielded significantly lower estimates of mean number of drinks consumed when heavier than usual drinking days is included in the Summary measure (124.0 vs 147.0 drinks), mean number of days drugs were used (9.3 vs 10.7) and mean number of episodes of anal intercourse with a primary partner (1.2 vs 2.2). Differences generally remained significant when assessed by length of time between the study interview and last use of alcohol or drugs, with the exception of number of anal sex episodes with primary partners. CONCLUSIONS: These findings indicate that Timeline estimates are lower than estimates using a more standard method (Summary measures). Discrepancies between these findings and those reported by other researchers indicate a need for further exploration of the effects of the mode of administration on various populations.     

Efficacy of relapse prevention: A meta-analytic review.

Although relapse prevention (RP) has become a widely adopted cognitive-behavioral treatment intervention for alcohol, smoking, and other substance use, outcome studies have yielded an inconsistent picture of the efficacy of this approach or conditions for maximal effectiveness. A meta-analysis was performed to evaluate the overall effectiveness of RP and the extent to which certain variables may relate to treatment outcome. Twenty-six published and unpublished studies with 70 hypothesis tests representing a sample of 9,504 participants were included in the analysis. Results indicated that RP was generally effective, particularly for alcohol problems. Additionally, outcome was moderated by several variables. Specifically, RP was most effective when applied to alcohol or polysubstance use disorders, combined with the adjunctive use of medication, and when evaluated immediately following treatment using uncontrolled pre–post tests. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A model for interpreting genetic correlations with estimates of parameters.

Presents a general additive model for kinship correlational structures which includes genetic, environmental, covariance, and measurement error components. The properties and assumptions of the model and other models which are viewed as special cases are discussed. A procedure is described for estimating the model parameters from sample correlation coefficients. The procedure results in numerical estimates of the ratio of genetic to environmental variance, estimates of the correlation between genetic and environmental factors, and estimates of the correlation between environments. A simple procedure for partitioning a correlation coefficient into its variance components is suggested. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The positive relationship between alcohol and heart disease in eastern Europe: potential physiological mechanisms

Research into the effect of alcohol on cardiovascular disease has indicated protective effects from moderate consumption. These observations, made in industrialized countries, have influenced policies on alcohol in countries where the situation may be quite different--specifically, where consumption is substantially higher or patterns of drinking are different. In central and eastern Europe and the former Soviet Union, a growing body of epidemiological research indicates a positive rather than negative association between alcohol consumption and cardiovascular deaths, especially sudden cardiac deaths. By means of a systematic review of published work, we examine whether there is a physiological basis for the observed association between alcohol and heart disease seen in eastern Europe, focusing on the effects of high levels of consumption and of irregular or binge drinking. In binge drinkers, cardioprotective changes in high-density lipoproteins are not seen, and adverse changes in low-density lipoproteins are acquired. Irregular drinking is associated with an increased risk of thrombosis, occurring after cessation of drinking. It predisposes both to histological changes in the myocardium and conducting system and to a reduction in the threshold for ventricular fibrillation. Measures of frequency as well as quantity of consumption should be included in epidemiological studies. Taken with the epidemiological evidence emerging from eastern Europe, these observations have important implications for estimates of the burden of disease attributable to alcohol.     

Hardy-Weinberg testing for continuous data

Estimation of allelic and genotypic distributions for continuous data using kernel density estimation is discussed and illustrated for some variable number of tandem repeat data. These kernel density estimates provide a useful representation of data when only some of the many variants at a locus are present in a sample. Two Hardy-Weinberg test procedures are introduced for continuous data: a continuous chi-square test with test statistic T(CCS) and a test based on Hellinger's distance with test statistic T(HD). Simulations are used to compare the powers of these tests to each other and to the powers of a test of intraclass correlation T(IC), as well as to the power of Fisher's exact test T(FET) applied to discretized data. Results indicate that the power of T(CCS) is better than that of T(HD), but neither is as powerful as T(FET). The intraclass correlation test does not perform as well as the other tests examined in this article.     

Using social indicators to predict addiction

Because of cost and other constraints, states often find it difficult to estimate need for treatment of alcohol-related problems from routine surveys. The social undesirability of illegal drug use makes the assessment of need for treatment of their use even more difficult. This paper uses independently obtained treatment need estimates to provide parameters for short-term prediction. We obtained the parameters by regressing the proportions of people addicted to alcohol (or drugs) in counties on social-indicator-based relative treatment need estimates for alcohol (or drugs). In addition to integrating estimates coming from independent sources, our approach presents an important tool for planning and resource allocation.