The relative power of training evaluation designs under different cost configurations.

After noting that the statistical power of training evaluation designs is a complex function of sample size, the reliability of the dependent measure, the correlation between pre- and posttest measures, and whether a randomized pretest–posttest or randomized posttest-only design is used, the authors show that the costs of conducting an evaluation are important considerations that also affect the relative power of the designs. Specifically, S costs, administrative costs, and item development costs are different components that can absorb resources when training evaluations are conducted. When total cost resources are fixed, these separate costs affect the relative power of pretest–posttest and posttest-only designs differently, and the posttest-only design may be the more preferred design under many different conditions. In other words, a variety of design and parameter tradeoffs affect power when total costs are fixed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Scholastic achievement of extension and regular college students.

In a course on personnel management, the learning achieved in a class of 39 day students was compared with that of a class of 39 evening students. Each class was given both a pretest and an identical posttest for each of two units of the course content. The pretest means of the day students did not differ significantly from the posttest means of the evening students; and the posttest means of the evening students differed from a chance score by less than an amount significant at the 5 per cent level of confidence. "In view of the obtained results, a question was raised about the advisability of granting college credit for work performed in evening off-campus extension courses." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relative importance of aptitude and prior domain knowledge on immediate and delayed posttests.

Two groups of college students were given a pretest covering basketball knowledge and asked to read a passage about basketball history. One group (IDP) took an immediate posttest of 27 items and a delayed posttest 1 wk later of the original items and 27 new items. The 2nd group (DPO) took only the delayed posttest of 54 items. Finally, a control group simply took the pretest and the 54-item posttest and scored at chance level on the posttest. Correlational analyses determined that, for the IDP group, pretest scores were significantly related to the immediate posttest. For the delayed posttest, both Scholastic Aptitude Test—Verbal subtest and pretest scores accounted for significant and unique portions of the variance. The latter results were replicated for the DPO group. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Power in randomized group comparisons: The value of adding a single intermediate time point to a traditional pretest-posttest design.

Adding a pretest as a covariate to a randomized posttest-only design increases statistical power, as does the addition of intermediate time points to a randomized pretest-posttest design. Although typically 5 waves of data are required in this instance to produce meaningful gains in power, a 3-wave intensive design allows the evaluation of the straight-line growth model and may reduce the effect of missing data. The authors identify the statistically most powerful method of data analysis in the 3-wave intensive design. If straight-line growth is assumed, the pretest-posttest slope must assume fairly extreme values for the intermediate time point to increase power beyond the standard analysis of covariance on the posttest with the pretest as covariate, ignoring the intermediate time point. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Contribution of pretesting to several measures of semantic desensitization effectiveness.

32 snake- or spider-phobic Ss were randomly assigned to 1 of 4 groups: pretest–posttest therapy, posttest-only therapy, pre–post control, or post-only control. Ss receiving semantic desensitization therapy showed less posttest anxiety on the semantic differential than controls regardless of testing condition. However, therapy Ss demonstrated less anxiety on a fear survey, a fear thermometer, and a behavioral approach test only in the pretest–posttest condition. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Developmental study of foreign language pronunciation.

106 4–9 yr olds were asked to pronounce French words that they heard pronounced by a native French speaker. On a pretest and a posttest, the experimenter said each word, and the S repeated it once. In a training session, the experimenter said each word, and the S repeated it 3 times. Pronunciations were scored for similarity to native French. Pronunciations were better on posttest than on pretest. There was a significant quadratic trend in the posttest scores; pronunciations improved with age up to 8 and 9 yrs and decreased thereafter. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improvements in horizontality performance as a function of type of training

90 undergraduates with low spatial perceptual ability were assigned to one of three groups of control, learner-guided instructions, and tutor-guided instructions. The groups differed in amount and type of spatial training administered in a laboratory setting. Participants completed horizontality and vertically spatial perceptual tasks immediately after pretesting and again one month later. Both groups given training showed more accurate performance at posttest than at pretest. In addition, the learner-guided instruction group outperformed the tutor-guided instruction group and control group at posttest and on novel but related tasks. Gender differences in posttest performance favoring males (n = 35) were obtained in the tutor-guided performance group and control group but not in the learner-guided instruction group. The number of hints the latter needed during training was predictive of posttest performance. Implications for assessment and remediation of undergraduates' spatial perceptual abilities are discussed.     

Transfer of task-switching training in older age: The role of verbal processes.

This study investigated the influence of verbal self-instructions (VSI) on the transfer of task-switching training in older adults (56–78 years). We applied an internally cued switching paradigm in a pretest–training–posttest design. Training-related improvements were not modulated by VSI. Transfer (the pretest–posttest reduction of switch costs) was most pronounced when participants applied the VSI at posttest after practicing the switching task without VSI. The results indicate that in contrast to transfer of executive control training, transfer of (verbal) strategy training seems to be limited and that VSI is most beneficial when the task-switching abilities are already well practiced. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence of motivational interviewing on explicit and implicit alcohol-related cognition and alcohol use in at-risk adolescents.

Both implicit and explicit cognitions play an important role in the development of addictive behavior. This study investigated the influence of a single-session motivational interview (MI) on implicit and explicit alcohol-related cognition and whether this intervention was successful in consequently decreasing alcohol use in at-risk adolescents. Implicit and explicit alcohol-related cognitions were assessed at pretest and one month posttest in 125 Dutch at-risk adolescents ranging in age from 15 to 23 (51 males) with adapted versions of the Implicit Association Test (IAT) and an expectancy questionnaire. Motivation to change, alcohol use and alcohol-related problems were measured with self-report questionnaires, at pretest, at posttest after one month, and at the six-month follow-up. Although the quality of the intervention was rated positively, the results did not yield support for any differential effects of the intervention on drinking behavior or readiness to change at posttest and six-month follow-up. There were indications of changes in implicit and explicit alcohol-related cognitions between pretest and posttest. Our findings raise questions regarding the use of MI in this particular at-risk adolescent population and the mechanisms through which MI is effective. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Alpha, beta, and gamma change in evaluation research: A structural equation approach.

Effects of organizational interventions are often studied within a nonequivalent control group design, with the pretest and posttest variables being self-report measures. Changes in these measures from pretest to posttest have multiple interpretations, depending on the processes believed to be responsible for the change. Golembiewski, Billingsley, and Yeager (1976) classified observed change in three categories as alpha, beta, or gamma change. Alpha change corresponds to absolute quantitative change. Beta change results from the respondent's subjective recalibration of the measurement scale. Gamma change results from the respondent's reconceptualization of the measured variable. A structural equation method is presented for the evaluation of intervention effects in the nonequivalent control group design when either alpha, beta, or gamma change may have occurred. The method uses an analysis of covariance in the latent variables underlying the pretest and posttest measures. Use of the method in distinguishing the three types of change, and linking them to the intervention, is discussed. The method is illustrated in examples using simulated data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of a verbal mediation strategy on cognitive processes in mathematics learning.

Hypothesized that the use of verbal rhymes (the form of mnemonics used) would facilitate learning and recall of mathematics computations and problem solving among 100 Japanese 7th graders. A pretest, posttest, and delayed posttest crossover design was employed to control for contamination between experimental and control groups and potential school and teacher effects. Ss received mathematics instruction for 10 wks on monomials, polynomials, equations, inequalities, and problem solving. Experimental Ss were taught with the aid of verbal rhymes. Results show no pretest differences between groups, but significant differences favoring experimental Ss on both the immediate and delayed (2-wk) posttest. It is concluded that the hypothesis was supported. (10 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Extraction treatment of a Class II, Division 1 malocclusion with anterior open bite with headgear and vertical elastics

Needlestick injuries, which lead to the transmission of hepatitis B, hepatitis C, and the AIDS virus, are a potentially serious threat to students during their clinical experiences. Exposure to infectious diseases, blood, and hazardous body fluids is one of the most frequently reported injury events by medical students at a health science center in the southwestern region of the United States. This study was conducted to determine the effectiveness of a customized intervention about infection control for second-year medical students (N = 200). Preparation for the intervention included a needs assessment, which included both qualitative and quantitative research methods that incorporated input from fourth-year medical students, medical staff members, and local hospital infection control specialists. The intervention included a pretest, a lecture, a demonstration of standard precautions and infection control procedure with 2 clinical scenarios, an exercise on proper handwashing, and a posttest. The evaluation of the intervention demonstrated a significant increase in posttest knowledge scores about infection control (from 12.6 +/- 2.1 pretest to 16.5 +/- 1.8 posttest, P < .001). Medical students showed a significant knowledge increase about infection control after participating in the intervention . Thus we recommend that all medical colleges and universities develop and evaluate a similar customized intervention for their medical students.     

Comparison of static and dynamic assessment procedures and their relation to independent performance.

Relationships between pretraining skills, ease of learning, and later posttest performance were assessed in both spatial (i.e., block design) and verbal (i.e., similarities) task domains. Eighty-four preschool children (M?=?4 years 11 months, range?=?4 years 0 months to 5 years 6 months) were given pretests, training, and posttests on block design and similarities tasks. Two measurement models and four structural models were tested to determine (a) if pretest, learning and posttest performances were domain specific, and (b) if paths from both pretests and learning assessments were needed to predict posttest performance. The measurement model that fit the data maintained separate verbal and spatial domains. The structural model that best fit the data included paths from both pretest and learning assessments to posttest performances within each domain. Thus, L. S. Vygotsky's (1978) claim that pretest and learning assessments together are better indicators of an individual's ability than either measure alone was supported. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Modification of concept identification performance in older adults.

A sequential training procedure combining operant and cumulative learning hierarchy principles was found effective in reducing the decrement in concept identification performance typically observed in older adults. 20 male and 20 female Ss aged 63-83 yrs were given a pretest and posttest, each involving a 3-category unidimensional concept identification problem. The training and reinforced-training Ss were given 3 training sessions between pretest and posttest. Training involved a programmed learning sequence designed to facilitate the development of an effective solution strategy. Control Ss were given only the pretest and posttest, while practice Ss were given 3 additional practice sessions. Substantial improvement was found in both the training and reinforced training groups, but only slight improvement obtained in the practice and control groups. Results are consistent with recent conceptions of cognitive and intellectual development in the older adults in which performance deficits are largely attributed to experiential factors. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The influence of the pretest on order effects in persuasive communications.

136 college students were assigned to 12 experimental groups arranged to examine the effects of taking an opinion pretest questionnaire and of delay of responses to a final posttest questionnaire. In other words order effects in opinion change were studied with opposed arguments used as the communications. The topic of the communication was Nikita Khrushchev. The results indicated a clear primary effect for groups not first exposed to a pretest. Primacy effects were also evident in groups where the delay between pretest and posttest was greatest. All other groups failed to yield directional effects of any kind. The relation of these results to the "set" hypothesis is then discussed. A 3-month follow up failed to yield a sufficient enough return of original Ss to draw other than highly tentative conclusions about the duration of order effects. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Cognitive strategies, experimenter modeling, and expectation in the attenuation of pain.

Obtained verbal reports of pain from 120 female undergraduates during a base-level pretest and also during a posttest conducted under 1 of several experimental treatments. The pain stimulus was a heavy weight applied to a finger for 2 min. During the posttest, Ss who had been asked to utilize cognitive strategies for reducing pain (i.e., to imagine pleasant events or to imagine the finger as insensitive) showed a reduction in pain as compared to uninstructed control Ss. Ss led to expect a reduction in pain, but not provided with cognitive strategies, also showed reduced pain during the posttest compared to controls, but the reduction was smaller than for Ss using cognitive strategies. An E modeling procedure, used with half of the Ss under each experimental treatment, was effective in reducing verbal reports of pain only for Ss with high pretest levels who were asked to imagine pleasant events. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Profiling the differences in students' knowledge, interest, and strategic processing.

The purpose for this research was threefold: to determine whether distinct and informative student profiles would emerge from knowledge, interest, and strategy measures specific to educational psychology; to compare these profiles with prior studies; and to explore changes in student profiles across an academic semester. As a result of cluster-analytic procedures, 3 distinct groups of participants emerged at pretest, and 4 emerged at posttest. One of the profiles that remained fairly consistent from pretest to posttest was the Learning-Oriented cluster. Students fitting this profile began the semester with the highest means in interest and strategic processing and with a moderate level of domain knowledge. By the end of the semester, this cluster obtained the highest means also on the domain knowledge test. Yet, the largest cluster at posttest seemed unable or unwilling to learn from demanding exposition. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of modeling, rehearsal, feedback, and remediation on acquisition of a counseling strategy.

Assessed the effects of 4 primary components of the microcounseling training model on the acquisition of a counseling strategy. The 4 training components investigated were (a) written and video models, (b) role-play practice, (c) peer feedback, and (d) role-play remediation practice. 40 beginning graduate students in counseling were assigned randomly to the following 4 treatment conditions: modeling; modeling and practice; modeling, practice, and feedback; and modeling, practice, feedback, and remediation. Written pretest, posttest, and retention test measures and posttest and retention test role-play interview measures were obtained of the dependent variable—the developing goals counseling strategy. Results show that, on the written measures, the performance of all 4 groups increased significantly from pre- to posttest and from pretest to retention test. A 1-way ANOVA revealed that all groups performed equally well on the posttest role-play interview. A repeated measures ANOVA on the role-play posttest and retention interviews indicated no significant differences between treatment groups or across trials. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of student performance in a multi-site clinical clerkship

A method for examining the equivalency of student performance in multi-site clinical clerkships is presented. Six student performance variables within the clerkship are explored: pretest, posttest, oral examination, presentation of a subject review, general clinical performance, and patient write-ups. Student achievement on two external examinations (a University of Washington School of Medicine third-year comprehensive examination and the Part II examination of the National Board of Medical Examiners) is reviewed and compared. This method applied to an obstetrics and gynecology clerkship at the University of Washington reveals no significant differences in student performance among sites on the pretest, oral examination, and both external examinations, but there were differences on four internal measures (posttest, presentations, write-ups, and clinical ratings). The use of this information for clerkship improvement and faculty development is described.     

Effects of counselor and peer-led group assertive training on Black adolescent aggression.

48 8th–9th grade Black males, selected for their aggressive classroom behavior, were randomly assigned to professional or peer counselor assertive training groups, professional or peer counselor discussion groups, and a no-treatment control group. All Ss were administered a pretest and posttest battery measuring assertive skill level, anger level, a projective assessment of aggression, and classroom aggressive behavior. Results suggest that professional counselors and peer counselors are equally effective in teaching assertive skills and that Ss who learn assertive responses will exhibit significantly less aggressive behavior. (14 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)