The effects of attrition on baseline comparability in randomized experiments in education: A meta-analysis.

Using meta-analysis, randomized experiments in education that either clearly did or clearly did not experience student attrition were examined for the baseline comparability of groups. Results from 35 studies suggested that after attrition, the observed measures of baseline comparability of groups did not differ more than would be expected given sampling error. The degree of either overall or differential attrition did not relate to baseline comparability, a finding that held under sensitivity analyses. Also, both overall and differential attrition rates were unrelated to posttest effect sizes. All of these analyses, however, lacked sufficient statistical power to detect small but potentially meaningful effects. Results suggest caution is warranted when applying quality scales and other blanket rules pertaining to attrition that are meant to either serve as inclusion-exclusion criteria or in scoring study quality. Much greater attention is needed to both the reporting of attrition in primary studies and to the development of conceptual and empirical models of the attrition process. These developments would aid further investigation of the relation between attrition and study outcomes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analysis of dichotomous variables in repeated measures experiments.

Provides an alternative to the method of L. A. Marascuilo and R. Serlin (1977) for the analysis of repeated measures experiments with dichotomous outcomes. Weighted linear regression methods are used to provide tests analogous to ANOVA. Tests of main effects and interactions are presented. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Individual differences methods for randomized experiments.

Experiments allow researchers to randomly vary the key manipulation, the instruments of measurement, and the sequences of the measurements and manipulations across participants. To date, however, the advantages of randomized experiments to manipulate both the aspects of interest and the aspects that threaten internal validity have been primarily used to make inferences about the average causal effect of the experimental manipulation. This article introduces a general framework for analyzing experimental data to make inferences about individual differences in causal effects. Approaches to analyzing the data produced by a number of classical designs and 2 more novel designs are discussed. Simulations highlight the strengths and weaknesses of the data produced by each design with respect to internal validity. Results indicate that, although the data produced by standard designs can be used to produce accurate estimates of average causal effects of experimental manipulations, more elaborate designs are often necessary for accurate inferences with respect to individual differences in causal effects. The methods described here can be diversely applied by researchers interested in determining the extent to which individuals respond differentially to an experimental manipulation or treatment and how differential responsiveness relates to individual participant characteristics. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Causal inference in randomized experiments with mediational processes.

This article links the structural equation modeling (SEM) approach with the principal stratification (PS) approach, both of which have been widely used to study the role of intermediate posttreatment outcomes in randomized experiments. Despite the potential benefit of such integration, the 2 approaches have been developed in parallel with little interaction. This article proposes the cross-model translation (CMT) approach, in which parameter estimates are translated back and forth between the PS and SEM models. First, without involving any particular identifying assumptions, translation between PS and SEM parameters is carried out on the basis of their close conceptual connection. Monte Carlo simulations are used to further clarify the relation between the 2 approaches under particular identifying assumptions. The study concludes that, under the common goal of causal inference, what makes a practical difference is the choice of identifying assumptions, not the modeling framework itself. The CMT approach provides a common ground in which the PS and SEM approaches can be jointly considered, focusing on their common inferential problems. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of psychosocial interventions with adult cancer patients: A meta-analysis of randomized experiments.

Meta-analytic methods were used to synthesize the results of published randomized, controlled-outcome studies of psychosocial interventions with adult cancer patients. Forty-five studies reporting 62 treatment control comparisons were identified. Samples were predominantly White, female, and from the United States. Beneficial effect size ds were .24 for emotional adjustment measures, .19 for functional adjustment measures, .26 for measures of treatment- and disease-related symptoms, and .28 for compound and global measures. The effect size of .17 found for medical measures was not statistically significant for the few reporting studies. Effect sizes for treatment-control comparisons did not significantly differ among several categories of treatment: behavioral interventions, nonbehavioral counseling and therapy, informational and educational methods, organized social support provided by other patients, and other nonhospice interventions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interactions for dichotomous variables in repeated measures designs.

Describes a method for analyzing interactions in a repeated measures design for a dichotomous dependent variable. The method is based upon the Cochran Q model but is restricted to the case in which a planned analysis is adopted. The model is illustrated for ordered and unordered independent variables. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A method for measuring generalization of imprinting effects in young chicks.

Describes 2 experiments, with a total of 180 Ross chicks as Ss, in which an attempt was made to construct a continuum of imprinting stimuli, together with a new method, the "stationary wheel," of estimating approach tendencies in young chicks. Ss were imprinted with 1 member of the continuum and either divided into subgroups, each of which was tested on 1 value of the continuum (Exp I), or tested on all values of the continuum (Exp II). Approach scores showed a significant decrement away from the training value at whichever end of the continuum chicks were trained. Results show that the stationary wheel method yielded replicable approach data without the disadvantages inherent in the "arena-type" method of measurement. Analyses of a number of other measures, including peep calls, twitter calls, and approach orientation, are also discussed. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sex effects in workplace justice outcomes: A field assessment.

Recent research on sex effects in workplace justice leads to different conclusions based largely on whether simulations or field assessments were relied on for experimental protocol. Our examination provides a field assessment that replicates in part and extends the investigation of dispute and disciplinary outcomes in the workplace. We argue that it is not merely the sex of the actor, but the nature of the dyadic composition (man/man, woman/woman, man/woman, woman/man) that is a critical influence on outcomes. This hypothesis is supported by significant differences in workplace justice outcomes based on the dyadic composition of the actors involved in dispute proceedings (N?=?498). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A randomized trial of extended telephone-based continuing care for alcohol dependence: Within-treatment substance use outcomes.

Objective: The study tested whether adding up to 18 months of telephone continuing care, either as monitoring and feedback (TM) or longer contacts that included counseling (TMC), to intensive outpatient programs (IOPs) improved outcomes for alcohol-dependent patients. Method: Participants (N = 252) who completed 3 weeks of IOP were randomized to up to 36 sessions of TM (M = 11.5 sessions), TMC (M = 9.1 sessions), or IOP only (treatment as usual [TAU]). Quarterly assessment of alcohol use (79.9% assessed at 18 months) was corroborated with available collateral reports (N = 63 at 12 months). Participants with cocaine dependence (N = 199) also provided urine samples. Results: Main effects favored TMC over TAU on any alcohol use (odds ratio [OR] = 1.88, CI [1.13, 3.14]) and any heavy alcohol use (OR = 1.74, CI [1.03, 2.94]). TMC produced fewer days of alcohol use during Months 10–18 and heavy alcohol use during Months 13–18 than TAU (ds = 0.46–0.65). TMC also produced fewer days of any alcohol use and heavy alcohol use than TM during Months 4–6 (ds = 0.39 and 0.43). TM produced lower percent days alcohol use than TAU during Months 10–12 and 13–15 (ds = 0.41 and 0.39). There were no treatment effects on rates of cocaine-positive urines. Conclusions: Adding telephone continuing care to IOP improved alcohol use outcomes relative to IOP alone. Conversely, shorter calls that provided monitoring and feedback but no counseling generally did not improve outcomes over IOP. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Clients' perceptions of therapists with high attrition rates.

59 adult clients assigned to high-dropout rate therapists and 51 assigned to low-dropout rate therapists rated the therapists on several demographic and therapy-process variables. Findings show that a number of the therapy-process variables significantly differentiated the 2 groups of therapists. It is advocated that researchers consider the role of the therapist and therapy-process dimensions in future studies of early psychotherapy termination. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Analysis of dichotomous variables in the presence of serial dependence.

[Correction Notice: An erratum for this article was reported in Vol 98(3) of Psychological Bulletin (see record 2009-20912-001). An unnecessary σ was added to the second line of Equation 32 on page 555. The second line should read: + 1/???](1 + ?a?b)/(1 – ?a?b), with the rest of the equation correct as printed.] Presents a model for the analysis of dichotomous data generated under a Markov chain model. Methods for estimating, testing, and comparing proportions and for analyzing dyadic interactions are described and compared with the standard techniques derived under the assumption of serial independence. It is shown that the regular test statistics must be adjusted by a correction factor that is a simple function of the within-S serial dependence parameter. Eigenvalues of the transition matrix are appended. (39 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of technical difficulties on learning and attrition during online training.

Although online instruction has many potential benefits, technical difficulties are one drawback to the increased use of this medium. A repeated measures design was used to examine the effect that technical difficulties have on learning and attrition from voluntary online training. Adult learners (N = 530) were recruited online and volunteered to participate in a 4-hr training program on using computer spreadsheets. Technical difficulties were inserted in some of the training modules in the form of error messages. Using multilevel modeling, the results indicated that the presence of these technical difficulties impaired learning, such that test scores were lower in modules where trainees encountered technical difficulties than in modules where they did not encounter technical difficulties. Furthermore, the effect on learning was greater among trainees who eventually withdrew from the course than among trainees who completed the course. With regards to attrition, pretraining motivation provided a buffer against dropping out, especially when trainees encountered technical difficulties. Learning also predicted attrition from the subsequent module, such that attrition was higher among trainees with low test scores in the previous module. The current study disentangles some of the implications of technical difficulties and suggests that organizations should provide trainees with the technical support required to overcome technical difficulties in training. Furthermore, the findings contribute to our theoretical understanding of the implications of interruptions on performance in online training. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation.

A meta-analysis of 128 studies examined the effects of extrinsic rewards on intrinsic motivation. As predicted, engagement-contingent, completion-contingent, and performance-contingent rewards significantly undermined free-choice intrinsic motivation (d?=?–0.40, –0.36, and –0.28, respectively), as did all rewards, all tangible rewards, and all expected rewards. Engagement-contingent and completion-contingent rewards also significantly undermined self-reported interest (d?=?–0.15, and –0.17), as did all tangible rewards and all expected rewards. Positive feedback enhanced both free-choice behavior (d?=?0.33) and self-reported interest (d?=?0.31). Tangible rewards tended to be more detrimental for children than college students, and verbal rewards tended to be less enhancing for children than college students. The authors review 4 previous meta-analyses of this literature and detail how this study's methods, analyses, and results differed from the previous ones. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A suggested effective method for unidirectional solidification under rotating magnetic field in the space experiments

In order to save costs and time during the unidirectional solidification experiments, it is necessary to obtain more and more information by solidifying one sample in a rotating magnetic field (RMF) to be performed on the space station. The macro- and microstructure developing during solidification is significantly influenced by the intensity and frequency of magnetic field, the solid/liquid interface velocity as well as the temperature gradient. A method by which one of the aforementioned four parameters can repeatedly be changed during the solidification is described. By this method, the transient phenomena occurring during parameters modification can also be observed. The results of earth experiments carried out by this method with Al7Si1Fe alloy are presented.     

A multivariate study of personality and academic factors in college attrition.

Performed multiple linear discriminant function analyses using academic, performance, and personality variables to discriminate among groups of transfers, persisters, and dropouts 2 yrs after college matriculation. 273 male and 433 female undergraduates were Ss. Measurements included the Personality Research Form, the MMPI, Block's Ego Control scale, grade point averages, and Scholastic Aptitude Test scores. Discrimination above base rates was obtained with either academic or personality variables. Analysis of all variables combined added no predictive power beyond that by either subset alone. Results held up on cross-validation. (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Couples' groups for parents of preschoolers: Ten-year outcomes of a randomized trial.

This paper reports the results of a 10-year follow-up of two variations of a couples' group preventive intervention offered to couples in the year before their oldest child made the transition to kindergarten. One hundred couples were randomly assigned to (1) a low-dose control condition, (2) a couples' group meeting for 16 weeks that focused more on couple relationship issues among other family topics, or (3) a couples' group meeting for 16 weeks that focused more on parenting issues among other family issues, with an identical curriculum to condition (2). Earlier papers reported that both variations of the intervention produced positive results on parent-child relationships and on the children's adaptation to kindergarten and first grade, and that the groups emphasizing couple relationships also had additional positive effects on couple interaction quality. The present paper uses growth curve analyses to examine intervention effects extending from the children's transition to kindergarten to the transition to high school—10 years after the couples groups ended. There were 6-year positive effects of the pre-kindergarten interventions on observed couple interaction and 10-year positive effects on both parents' marital satisfaction and the children's adaptation (hyperactivity and aggression). Discussion includes a focus on the implications of these results for family policy, clinical practice, and the need to include a couples focus in preventive interventions to strengthen family relationships and enhance children's adaptation to school. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A problem with the interpretation of before-and-after experiments of drug tolerance.

Four groups of amygdala-kindled rats were exposed lo 15 daily tolerance-development trials. On each trial, 1 group received diazepam (2.5 mg/kg IP) 1 hr before a convulsive stimulation, 1 group received diazepam 1 hr after a stimulation, 1 group received 15 diazepam injections but no stimulation, and a combined control group received 15 vehicle injections either with or without a stimulation. Consistent with previous findings, only subjects that had received diazepam before the stimulations during the tolerance-development phase displayed significant tolerance to diazepam's anticonvulsant effect on the first test trial. The major new finding was that diazepam injections by themselves induced significant savings (i.e., residual tolerance) on the test trials but that diazopam injections preceded by convulsive stimulations did not. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Issues for covariance analysis of dichotomous and ordered categorical data from randomized clinical trials and non-parametric strategies for addressing them

Analysis of covariance is an effective method for addressing two considerations for randomized clinical trials. One is reduction of variance for estimates of treatment effects and thereby the production of narrower confidence intervals and more powerful statistical tests. The other is the clarification of the magnitude of treatment effects through adjustment of corresponding estimates for any random imbalances between the treatment groups with respect to the covariables. The statistical basis of covariance analysis can be either non-parametric, with reliance only on the randomization in the study design, or parametric through a statistical model for a postulated sampling process. For non-parametric methods, there are no formal assumptions for how a response variable is related to the covariables, but strong correlation between response and covariables is necessary for variance reduction. Computations for these methods are straightforward through the application of weighted least squares to fit linear models to the differences between treatment groups for the means of the response variable and the covariables jointly with a specification that has null values for the differences that correspond to the covariables. Moreover, such analysis is similarly applicable to dichotomous indicators, ranks or integers for ordered categories, and continuous measurements. Since non-parametric covariance analysis can have many forms, the ones which are planned for a clinical trial need careful specification in its protocol. A limitation of non-parametric analysis is that it does not directly address the magnitude of treatment effects within subgroups based on the covariables or the homogeneity of such effects. For this purpose, a statistical model is needed. When the response criterion is dichotomous or has ordered categories, such a model may have a non-linear nature which determines how covariance adjustment modifies results for treatment effects. Insight concerning such modifications can be gained through their evaluation relative to non-parametric counterparts. Such evaluation usually indicates that alternative ways to compare treatments for a response criterion with adjustment for a set of covariables mutually support the same conclusion about the strength of treatment effects. This robustness is noteworthy since the alternative methods for covariance analysis have substantially different rationales and assumptions. Since findings can differ in important ways across alternative choices for covariables (as opposed to methods for covariance adjustment), the critical consideration for studies with covariance analyses planned as the primary method for comparing treatments is the specification of the covariables in the protocol (or in an amendment or formal plan prior to any unmasking of the study.