Predictive value of letters of recommendation vs questionnaires for emergency medicine resident performance

OBJECTIVE: To evaluate the predictive value of standard letters of recommendation (LORs) vs preprinted questionnaires (PPQs) for resident performance at one emergency medicine (EM) residency program. METHODS: A retrospective association of LORs and PPQs with in-training residents performance ratings was done at one EM residency program. The residency application files of EM residents who completed the program were reviewed to locate files that had LORs and PPQs written by the same author. Seventeen resident files contained 32 LOR/PPQ pairs. These LORs and PPQs were submitted in a blinded fashion to 3 outside EM residency directors. Each LOR and PPQ was evaluated for the applicant's suitability for the specialty of EM, medical knowledge, procedural skills, interpersonal skills, motivation, and overall rank. The scores given by the outside reviewers were compared with resident performance ratings determined by 5 EM attending physicians who evaluated the residents along the same 6 dimensional ratings. RESULTS: Statistically, no differences were found between the LORs and PPQs in predicting resident performance. CONCLUSIONS: PPQs may substitute for LORs in the evaluation of resident applicants.     

Effect of teacher sex, student sex, and teacher warmth on the evaluation of college instructors.

Examined the influence of teacher sex, student sex, and teacher warmth as perceived by students and teachers on teacher evaluation. Male and female instructors in the same department were matched on the level of course taught. 22 pairs of courses evaluated by 838 college students were obtained. Twenty Instructional Improvement Questionnaire items that directly evaluate instructor performance were analyzed using a 3-factor analysis of variance and the .0025 level of significance. No interactions between faculty sex, student sex, and teacher warmth were found. When Ss rated their instructor's interest and warmth, teachers who were warmer and primarily interested in students received higher ratings in teaching effectiveness. When teachers rated themselves on warmth and interest, self-ratings interacted with faculty sex. Generally, female teachers received higher effectiveness ratings than did male teachers when they considered themselves low in warmth or interested in course content. Male teachers who rated themselves high in warmth or primarily interested in students received higher ratings than did male teachers who rated themselves low in warmth or primarily interested in course content, respectively. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit

OBJECTIVE: Compare resident evaluations by self, nurses, and attending physicians. DESIGN: Prospective cohort. SETTING: University intensive care unit. SUBJECTS: Sixty residents. INTERVENTIONS: End-rotational evaluation using a standardized, multiple-choice examination and one of two subjective instruments, Global Rating Scale and Behaviorally Anchored Rating Scale. MEASUREMENTS AND MAIN RESULTS: Means for overall competence, using both the Behaviorally Anchored Rating Scale and the Global Rating Scale clustered between 3 to 4 on a 5-point scale. Physicians' evaluations correlated with the multiple-choice test scores (Spearman's rho 0.3082, p = .005, n = 82), whereas neither self-evaluation (Spearman's rho 0.1124, p = .65, n = 42) nor nurses' evaluations (Spearman's rho 0.2060, p = .069, n = 79) had a significant correlation with test scores. Spearman's correlations were not significant for either overall competence or specific medical knowledge by any category of evaluator using the Global Rating Scale. Spearman's rho correlations and kappa statistic between the three types of evaluators (physicians, nurses, and self) for each criterion of the Behaviorally Anchored Rating Scale demonstrated significant correlations between the ratings of physicians and nurses, except for the assessment of humanistic qualities. Pooled clinical skills-history taking (b = 0.277, p <.009), humanistic qualities (b = 0.607, p <.000), and professional attitudes and behavior (b = 0.488, p < .000) systematically differed in ratings comparing self with nurse and physician (by analysis of variance). The explanatory power of the model of ratings (independent variables of year of residency, category of evaluator, evaluation criteria, and interaction terms) was 47.3% (r2adj). CONCLUSIONS: Self-rating by residents did not correlate to multiple-choice test scores and differed in some criteria with physicians' or nurses' evaluations. We found many similarities and some differences between physicians' and nurses' evaluations of residents. We speculate that different categories of evaluators assess different aspects of performance. Assessment by a varied group of evaluators should be used when attempts to predict future practice are made.     

A randomized trial of physicians and physical therapists as instructors of the musculoskeletal examination

OBJECTIVE: To evaluate the effectiveness of physicians vs physical therapists as instructors of the musculoskeletal examination to second year medical students. METHODS: A randomized trial conducted over 3 consecutive years in a physical diagnosis course at The University of North Carolina. During the first (baseline) year, medical students received education about the musculoskeletal examination from a lecture and supervision by clinical preceptors. This increased in the second and third (intervention) years where random halves of each class received supplementary clinical instruction which included a structured manual, a videotape, and supervised practice with either general internal medicine physicians or physical therapists. Outcomes were measured from student performance on a practical test of clinical skills, and by written student evaluations of the supplementary instruction. RESULTS: As measured on the clinical examination, performance during the intervention years improved over that during the baseline year in both intervention groups. Measures of medical students' clinical skills did not correlate with measures of academic aptitude. CONCLUSIONS: Clinical education about the musculoskeletal examination should be structured and systematic. Medical students prefer clinical education that is active, provides clear directions, and gives them performance feedback.     

Midterm feedback from students: Its relationship to instructional improvement and students' cognitive and affective outcomes.

Feedback from student ratings collected at midterm resulted in (a) more favorable student ratings at the end of the term, (b) better final examination scores, and (c) more favorable affective outcomes. Ss were the 931 students in 30 sections of a course in computer programming. Randomly selected instructors were given feedback from midterm ratings and met with the authors to discuss the ratings and strategies for improvement. Ss in the feedback sections were similar to other Ss in terms of pretest scores and midterm ratings. These findings and a review of the literature suggest that feedback from student ratings, particularly when coupled with a frank discussion of their implications with an external consultant, can be an effective intervention for improving teaching effectiveness. (13 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Impact of work experience on the validity of student evaluations of teaching effectiveness.

Examined whether student evaluations of teaching effectiveness changed significantly once the students had left the university and had gained work experience. Students in their final year at a university were asked to grade lecturers on 9 items relating to teaching effectiveness and course effectiveness; 723 students responded. After the students had graduated and had been working for 1 yr, they were asked again to give a rating of teaching effectiveness for their former lecturers and to nominate the "best" and "worst" teachers; 506 replied. Comparisons of the results show that there was little difference in the 2 ratings of lecturers by their teaching scores. Increasing maturity and job experience did not significantly alter student evaluations of relative teaching effectiveness. (22 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effect of evaluator and resident gender on the American Board of Internal Medicine evaluation scores

OBJECTIVE: To examine the effects of resident and attending physician gender on the evaluation of residents in an internal medicine training program. DESIGN: Cross-sectional study. SETTING: Large urban academic internal medicine residency program. PARTICIPANTS: During their first 2 years of training, 132 residents (85 men, 47 women) received a total of 974 evaluations from 255 attending physicians (203 men, 52 women) from 1989 to 1995. MEASUREMENTS: The primary measurements were the numerical portions of the American Board of Internal Medicine evaluation form. Separate analyses were performed for each of the nine evaluation dimensions graded on a scale of 1 to 9. The primary outcome was the difference in the average scores received by each resident from male versus female attending physicians. RESULTS: Compared with female trainees, male residents received significantly higher scores from male attending physicians than from female attending physicians in six of the nine dimensions: clinical judgment, history, procedures, relationships, medical care, and overall. Similar trends, not reaching conventional levels of statistical significance, were observed in the other three categories: medical knowledge, physical exam, and attitude. These differences ranged from 0.24 to 0.60 points, and were primarily due to higher grading of male residents by male attending physicians than by female attending physicians. CONCLUSIONS: In one academic training program, we found a significant interaction in the grading process between the gender of internal medicine residents and the gender of their attending evaluators. This study raises the possibility that subtle aspects of gender bias may exist in medical training programs.     

Case 6--1998. Perioperative anesthetic management of a patient presenting for resection of a tracheal mass

PURPOSE: Relatively little research has focused on faculty development methods that assist basic science teachers to improve their instructional skills. This study was designed to assess the effectiveness for basic science faculty of a faculty development seminar series that had been previously shown useful for clinical teachers. METHOD: The Stanford Faculty Development Program's seminars on clinical teaching were adapted for basic science instruction. Eight pathology faculty participated in a series of nine small-group seminars designed to provide teachers with knowledge of a framework for analyzing teaching and identifying areas for improvement, and skill-based training in specific teaching behaviors. Each seminar included (1) brief lectures, (2) review of videotaped reenactments of teaching interactions, (3) role-play exercises with videotape review, and (4) formulation of personal and departmental teaching goals. RESULTS: Program evaluation included multiple measures: participant self-assessment, student ratings of the participants, and blinded ratings of pre- and post-seminar videotapes of participants' classroom teaching. All measures indicated a positive effect of the intervention. CONCLUSION: Faculty development programs have significant potential to enhance basic science instructors' teaching effectiveness.     

Attributions of ability and motivation in a natural setting.

Examined the effects of different patterns of student performance on teachers' attributions of ability and motivation in a school setting. 415 6–8th graders and 17 teachers served as Ss. Lorge-Thorndike Intelligence Test scores and grades affected attributions of ability and motivation in an additive fashion. In contrast to F. Heider's (see PA, Vol 33:971) proposition, attributions of ability and motivation had a positive relationship when performance was controlled. Teachers had greater confidence in their attributions when performances were extreme, particularly in a positive direction. Although consistency in performance between test scores and grades did not affect their confidence, teachers were more confident when students' grades were consistent with the teachers' perceptions of the children's ability. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The teacher and student as Pygmalions: Joint effects of teacher and student expectations.

Examined the effects of teachers' expectations about students and students' expectations about teachers on the performance and attitudes of both participants. 60 female undergraduates were designated as teachers and were led to expect a high- or low-ability student, and a further 60 Ss acting as students were independently led to expect a teacher of low or high competence. Teachers and students were then randomly paired in a teaching session. Results show that student performance was a function of the teacher's expectations. In addition, teachers' attitudes and rated competence were affected by their expectation regarding the student, and students' attitudes were affected by their expectation about the teacher. (10 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Resident and faculty perceptions of effective clinical teaching in family practice

Increasing interest in clinical teaching has led to the realization that the unique subset of skills which characterizes effective clinical teaching needs to be identified. Such identification will lead to development of these skills and improvement in the quality of clinical teaching. Family practice faculty are vitally concerned with improving their clinical teaching skills, since clinical teaching is the core of education in family medicine and since many family physicians who become preceptors have had no formal training as teachers. In this investigation of effective clinical teaching behaviors, faculty and residents generally agree in their perceptions of the helpfulness of 58 clinical teaching behaviors. Neither group felt that emphasis on references and research is as important a factor in effective clinical teaching as are residents' active participation in the learning situation and positive preceptor attitudes toward teaching and residents. It was perceived that the ineffective clinical teacher has a negative attitude toward residents, is inaccessible, and lacks skills in providing feedback, while the effective clinical teacher has skills in two-way communication, creates an educational environment that facilitates learning, and provides constructive feedback to residents.     

Pharmacist influence on economic and morbidity outcomes in a tertiary care teaching hospital

The influence of pharmacist participation on economic and morbidity outcomes at a tertiary care teaching hospital was studied. Patients admitted to internal medicine wards during a nine-month period were assigned to either a treatment team or a control team. Each team consisted of an attending physician, senior and junior medical residents, and medical students; the treatment team included a pharmacist who reviewed all patient charts, made rounds with the team, and recommended modifications of drug therapy. Pharmacy interaction with the control team was limited to contacting physicians about potentially dangerous orders, answering questions from the medical team, and handling orders for items not on the formulary or otherwise unavailable. After discharge, data from patient records were analyzed for pharmacy costs and total hospital costs and length of stay (as markers of the pharmacist's effect on economics and morbidity, respectively). Analysis of baseline characteristics showed that the two groups of patients were statistically comparable. Treatment team patients who were included in the data analysis (414) had significantly shorter stays (by a mean of 1.3 days) and lower pharmacy and total hospital costs (by a mean of $301 and $1654, respectively) than those included in the control team analysis (453). The direct participation of a pharmacist on a patient care team significantly decreased pharmacy and hospital costs, as well as length of stay, compared with minimal participation of a pharmacist.     

Teachers' and deaf students' perceptions of communication ease and engagement.

Collected ratings of communication ease and of engagement for 95 deaf junior and senior high school students (aged 12–21 yrs) in a large, urban, multiethnic school for the deaf. Ratings were made by the students' English teachers and by the students themselves. High teacher ratings for communication ease and engagement were associated with high academic achievement as measured by the Stanford Achievement Test. High student ratings for communication ease were also associated with academic achievement. The 2 best predictors of academic achievement were student-rated communication ease and teacher-rated engagement. Findings suggest that it is important for teachers to be sensitive to the level of communication comfort of their students because students who enjoy easier communication may be more likely to be engaged, or more actively involved, in classroom learning and to demonstrate higher academic achievement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The moderating effects of attachment style on students' experience of a transition to university group facilitation program.

This study investigated how attachment style predicted first-year students' responses to a transition to university group facilitation program. Participants (N = 148) at three universities were randomly assigned to a control or facilitation group. Each group consisted of on average six first-year students and two facilitators who met weekly for nine weeks at the beginning of the first semester to discuss issues relevant to the transition to university. Attachment styles (preoccupied, secure, dismissing, and fearful), feelings toward group experiences, and adjustment were assessed by questionnaire at the end of the program in November and again in March. Results indicated that, as hypothesised, dismissing style scores were generally negatively correlated with student attendance and feelings toward the group. Secure style ratings tended to be positively correlated with reports of group feelings. Adjustment measures collected in March indicated a potential buffering effect of the facilitation program for the preoccupied students in that the preoccupied students in the control group indicated more depression and loneliness, plus poorer adjustment to university, than the preoccupied students in the intervention groups. However, all other styles, including the dismissing style, did not show differences between intervention and control groups on subsequent adjustment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Correspondences among parent, teacher, and student perceptions of adolescents' learning disabilities

A group of 26 adolescents with learning disabilities (Grades 9 through 12), their parents, and their special education teachers were asked to rate the students' skills in each of 21 specific areas covering general ability, oral language, reading, written language, math, study skills, motivation, social skills, attention, and nonverbal skills. Correspondences in the absolute and relative ratings of parents, teachers, and students across the 21 skill areas were examined. The parents' ratings were consistent with those of the teachers in 16 areas and significantly lower than the teachers in 5 areas. The students' ratings were generally higher than those of their parents and teachers. The student-teacher differences were significant in 6 areas, whereas the student-parent differences were significant in 11 areas. Although generally lower in absolute terms than the ratings of their children, the parents' relative ratings were strikingly parallel to their children's ratings across skill areas (r = 80). Differences in the reference groups used for the ratings did not seem to account for the discrepant ratings. Possible implications of the differing perceptions of students' learning disabilities for students' self-esteem and academic progress are discussed.     

Construct validation of a strategy model of student self-regulated learning.

Using student interviews, teacher ratings, and achievement test outcomes, we validated a strategy model of student self-regulated learning as a theoretical construct. Forty-four male and 36 female high school students were asked to describe their use of 14 self-regulated learning strategies in six contexts, and their teachers rated these students for their self-regulated learning during class. Factor analyses of the teachers' ratings along with students' scores on a standardized test of mathematics and English revealed a single self-regulated learning factor that accounted for nearly 80% of the explained variance and two smaller factors that were labeled Student Verbal Expressiveness and Achievement. Students' reports of using self-regulated learning strategies during a structured interview correlated .70 with the obtained teachers' rating factor and were negatively related to the Student Verbal Expressiveness and Achievement factors. Our results indicate both convergent and discriminative validity for a self-regulated learning construct. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Classroom Management in Elementary Mainstreaming Classrooms.

The goals of this research were (a) to determine if teachers who were trained in the Utah State University Classroom Management Program changed their use of specific skills more than did comparable control teachers and (b) to determine if students in experimental classrooms made greater changes in ontask and deviant behavior than did comparable students in control classrooms. The study employed a control group design with pre- and postobservations of teacher and student behavior. A total of 39 elementary school teachers in mainstreaming classrooms were randomly assigned to two groups. Experimental teachers were trained in the classroom management program, whereas the control teachers were trained in a similar program of the same duration that covered different teaching skills. The student sample consisted of 322 students selected from participating classrooms and included subsamples of handicapped and minority students. Analysis of covariance of 16 teacher behaviors found that experimental teachers were superior to control teachers in nine behaviors. Experimental students obtained significantly more favorable posttreatment means than did control students for on-task, mildly deviant, and seriously deviant behavior. Most comparisons between handicapped and racial student subgroups were not significant. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of student ethnicity on judgments of ADHD symptoms among Hispanic and White teachers.

The present study examined whether teacher ratings of student disruptive, impulsive, and inattentive behaviors vary according to teacher-student ethnic differences. A total of 129 Hispanic and 89 White teachers observed standardized videotapes of a Hispanic and a White child and assessed each child for hyperactive-inattentive behaviors, using a rating scale. Based on analyses of subscales, this study revealed significant differences in ratings of attention-deficit/hyperactivity disorder (ADHD) behaviors among Hispanic and White teachers with Hispanic teachers reporting higher mean scores on the Hyperactivity-Impulsivity Scale. This was true on the ratings of the Hispanic student but not on the White student. Further analyses suggested that when acculturation was covaried, no significant group differences remained, suggesting that teacher perception of deviance may be partially mediated by cultural values more than ethnicity. The need to consider the rater's ethnicity and ethno-cultural variables is discussed, and findings are explained in terms of the adult distress threshold model. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Cognitive and personality characteristics of high school students as predictors of the way they are rated by their teachers: A longitudinal study.

Studied longitudinally, over 6 yrs, the relationship between students' cognitive and personality characteristics, teacher judgments of students, and academic performance. Ss were 142 Caucasian students age 12 yrs, 8 mo originally and 17 yrs, 8 mo when the study was completed. Measures included teacher ratings of student traits, WISC and WAIS subtests, and the Personality Research Form. Results indicate that teacher ratings of students had differential meaning according to the grade level in which they were obtained. 2 possible explanations of the data are given: (a) in making judgments of students, teachers attend to different trait configurations of students depending on the degree of evolution of the teacher-student relationship; and (b) findings may be a function of developmental factors in the students. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)