Student, peer, and self evaluations of college instructors.

Compared student, peer (or colleague), and self ratings in terms of item statistics, convergent and discriminant validity, and relation to student learning. Ratings from the 3 sources (involving 263 students and 14 instructors) were similar in range and distribution, although colleagues tended to give the most favorable ratings, students the least favorable. Individual student and colleague reliabilities were also similar; composite student reliabilities were considerably higher than composite colleague reliabilities, only partly because of differing sample sizes. Student and self ratings and rankings were quite good in terms of convergent and discriminant validity, but no student, peer, or self rating was significantly related to residualized student achievement. (32 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Teacher personality traits and student instructional ratings in six types of university courses.

Colleague ratings of 29 personality traits were studied in relation to student ratings of teaching effectiveness in a sample of 46 psychology teachers. Instructors were evaluated in six different types of university courses, ranging from freshman lecture classes to graduate research seminars. Major findings were as follows: (1) Rated teaching effectiveness varied substantially across different types of courses for a given instructor; (2) teaching effectiveness in each type of course could be predicted with considerable accuracy from colleague ratings of personality; and (3) the specific personality traits contributing to effective teaching differed markedly for different course types. It was concluded that psychology instructors tend to be differentially suited to different types of courses and furthermore that the compatibility of instructors to courses is determined in part by personality characteristics. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality, classroom behavior, and student ratings of college teaching effectiveness: A path analysis.

Tested the hypothesis that classroom teaching behavior mediates the relation typically found between personality and college teaching effectiveness. Colleagues rated 37 full-time college instructors on 29 personality traits, and trained observers assessed the frequency with which the same instructors exhibited 95 specific classroom teaching behaviors. Instructional effectiveness was measured by global end-of-term student ratings averaged over a 5-yr period. Path analyses revealed that approximately 50% of the relation between personality and teaching effectiveness was mediated by classroom behavior. The highly rated teacher was found to exhibit 2 types of personality traits: achievement orientation and interpersonal orientation. Results are discussed in terms of the validity of student ratings of teaching and in relation to M. J. Dunkin and B. J. Biddle's (1974) model of classroom teaching. (31 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Low-inference classroom teaching behaviors and student ratings of college teaching effectiveness.

Six to 8 trained observers visited classes (3 classes/lecturer) taught by 54 university lecturers receiving either low, medium, or high student ratings. The observers, using the Teacher Behaviors Inventory, estimated the frequency of occurrence of 60 specific, low-inference teaching behaviors. Significant differences among low-, medium-, and high-rated Ss were found for 26 individual behaviors divided among 7 categories of teaching. Group differences were largest for attention-getting behaviors such as speaking expressively, moving about while lecturing, using humor, and showing enthusiasm for the subject. Factor analysis of individual teaching behaviors yielded 9 interpretable factors, of which three (Clarity, Enthusiasm, and Rapport) differed significantly across groups, and all but one showed correlations with various teacher and course characteristics. Results are discussed with reference to the pivotal role of attention-getting behavior in classroom teaching, the validity of student instructional ratings, and the design of teaching improvement programs in higher education. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding parent-adolescent interactions: Perceptions of family members and outsiders.

To see whether adolescents view parent–child relationships more like their parents or more like outsiders, 41 mother–father–adolescent child triads were videotaped interacting over two tasks. Each family member then viewed both interactions and rated levels of anxiety, dominance, involvement, and friendliness, for each interactant. These same interactions were also watched and rated by another mother–father–adolescent triad (outsider family) who did not know the family. These interactions were then rated by a trained observer. For ratings of anxiety, dominance, involvement, and friendliness, analyses involved comparisons (a) between the insider and outsider families ratings, on correlations between insider family members and outsider family members with the trained observer, and (b) comparisons among the three types of observers. Ratings by members of the outsider family were generally more negative than those by members of the insider family. Ratings of self tended to be more objective than ratings of other family members. Ratings by the trained observer were more highly correlated with those of the outsider family, although differences between the means showed that trained observers used a similar metric to the insider family. The importance of looking at patterns of ratings as well as mean differences is emphasized. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Methodological considerations in the evaluation of a teacher-training program.

Investigated the effects of 3 types of teacher training on graduate teaching assistants' (TAs) teaching behavior, perception of teaching ability, and ratings of effectiveness by their students. In addition, the effects of response-shift bias, an instrumentation-related problem, were examined to ascertain if this source of contamination was operating in the evaluation of this educational program. 38 TAs participated in the training. The training produced significant changes in the trained TAs' teaching behavior, self-ratings of teaching ability, and student ratings of instruction relative to the no-treatment control group. Response-shift bias was noted in the self-reports and controlled through the collection of retrospective pretests. Implications and methodological considerations for future teacher-training programs are discussed. (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Impact of work experience on the validity of student evaluations of teaching effectiveness.

Examined whether student evaluations of teaching effectiveness changed significantly once the students had left the university and had gained work experience. Students in their final year at a university were asked to grade lecturers on 9 items relating to teaching effectiveness and course effectiveness; 723 students responded. After the students had graduated and had been working for 1 yr, they were asked again to give a rating of teaching effectiveness for their former lecturers and to nominate the "best" and "worst" teachers; 506 replied. Comparisons of the results show that there was little difference in the 2 ratings of lecturers by their teaching scores. Increasing maturity and job experience did not significantly alter student evaluations of relative teaching effectiveness. (22 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of students' evaluations of college teaching: A multitrait–multimethod analysis.

College instructors in 329 classes evaluated their own teaching effectiveness with the same 35-item rating form used by their students. There was student–instructor agreement in courses taught by teaching assistants (r?=?.46), undergraduate courses taught by faculty (r?=?.41), and graduate level courses (r?=?.39). Separate factor analyses of the student and instructor ratings demonstrated that the same 9 evaluation factors (e.g., work load, organization, interaction) underlay both sets of ratings. A multitrait–multimethod analysis supported convergent and divergent validity of these rating factors. Not only were correlations between student and instructor ratings on the same factors statistically significant for each of 9 factors, but correlations between their ratings on different factors were low. Findings demonstrate student–instructor agreement on evaluation of teaching effectiveness, support the validity of student ratings for both graduate and undergraduate courses, and emphasize the importance of using multifactor rating scales derived through the application of factor analysis. (28 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Hamilton Rating Scale for Depression: Reliability and validity of judgments of novice raters.

Reliability studies of the Hamilton Rating Scale for Depression (HRSD) in therapy-outcome research require at least 2 clinicians. The present authors hypothesized that a less costly alternative, such as using trained undergraduates as 2nd raters, would produce results comparable to the use of 2 clinicians. Four expert raters provided criterion ratings for the HRSD on 20 depressed women. Three trained undergraduates rated the same Ss. The expert and student raters both made reliable ratings on the HRSD. Estimates of criterion validity for the student raters were also in the satisfactory range. (9 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Student evaluations of teaching measure the intervention, not the effect.

Comments on the article by W. J. McKeachie (see record 1997-43129-006) that discusses 4 additional articles in the November 1997 issue of American Psychologist concerning student evaluations of teaching. The current author does not comment on the consensus belief that certain forms of student ratings are useful for improving teaching. Rather, he addresses the dangerous entrenched doctrine that considers student ratings or measurements of teaching behaviors to be the best way to measure or define teaching effectiveness. An mathematical alternative to relying on students' evaluations is presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity concerns and usefulness of student ratings of instruction.

The validity of student rating measures of instructional quality was severely questioned in the 1970s. By the early 1980s, however, most expert opinion viewed student rating measures as valid and as worthy of widespread use. In retrospect, older discriminant-validity concerns were not so much resolved as they were displaced from research attention by accumulating evidence for convergent validity. This article introduces a Current Issues section that gives new attention to validity concerns associated with student ratings. The section's 4 articles deal, respectively, with (a) conceptual structure (are student ratings unidimensional or multidimensional?), (b) convergent validity (how well do ratings correlate with other indicators of effective teaching?), (c) discriminant validity (are ratings influenced by factors other than teaching effectiveness?), and (d) consequential validity (are ratings used effectively in personnel development and evaluation?). Although all 4 articles favor the use of ratings, they disagree on controversial points associated with interpretation and use of ratings data. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

How useful are student ratings? Reactions to comments on the current issues section.

Replies to comments on articles by J. S. Armstrong, D. Buck, J. Friedrich, and R. E. Redding (see records 1998-11971-007; 1998-11971-008; 1998-11971-009; and 1998-11971-010, respectively) regarding the usefulness of student ratings of teaching. It is noted that the articles take different approaches to a shared form of argument: If student ratings are elevated by instructor behavior that does not contribute to learning, then the use of student ratings to evaluate faculty can encourage this nonproductive teaching behavior. Student ratings can thus defeat, rather than promote, instructional quality. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Do grading standards affect student evaluations of teaching? Some new evidence on an old question.

The relationship between the grade a student receives and the evaluation the student gives to the instructor has been a topic of concern and extensive investigation in educational research. Ideally, teaching evaluations should be based on teaching effectiveness and not on leniency of grading standards. Previous studies using a variety of methodologies and statistical analyses have found inconclusive results. The present study, after factor analyzing teacher evaluations by 5,878 graduate and undergraduate students into 3 dimensions, found significant relationships between expected grade and the dimensions of Course Effectiveness, Teacher Effectiveness, and Course Difficulty. The 1st set of relationships is viewed as support for a nonbias interpretation of why grades and ratings correlate, whereas the latter 2 sets support a bias interpretation. Correlations between grades and evaluations varied across types of courses, with required courses showing the strongest correlations between grades and ratings. Since these courses are typically less popular than electives, grading leniency may have a greater influence on course evaluations. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Midterm feedback from students: Its relationship to instructional improvement and students' cognitive and affective outcomes.

Feedback from student ratings collected at midterm resulted in (a) more favorable student ratings at the end of the term, (b) better final examination scores, and (c) more favorable affective outcomes. Ss were the 931 students in 30 sections of a course in computer programming. Randomly selected instructors were given feedback from midterm ratings and met with the authors to discuss the ratings and strategies for improvement. Ss in the feedback sections were similar to other Ss in terms of pretest scores and midterm ratings. These findings and a review of the literature suggest that feedback from student ratings, particularly when coupled with a frank discussion of their implications with an external consultant, can be an effective intervention for improving teaching effectiveness. (13 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Construct validity of multiple measures in teaching, research, and service and reliability of peer ratings.

The construct validity measures in teaching, research, and service were determined using a multitrait–multimethod matrix. The measures in research (research publications, research grants, research awards, and peer ratings in research) and three of the measures in teaching (student evaluations, teaching awards, and peer ratings in teaching) supported convergent and divergent validity. However, the three-pronged model in academia was not validated, and exploratory factor analysis identified 5 and possibly 6 domains of behavior (research, classroom teaching, writing about teaching, community service, writing about service, and internal service). For one measure, peer ratings, reliability varied across academic areas and was moderately high for only high-confident ratings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Student evaluations of college professors: Are female and male professors rated differently?

Over 1,000 male and female college students of 16 male and female professors (matched for course division, years of teaching, and tenure status) evaluated their instructors in terms of teaching effectiveness and sex-typed characteristics. Male students gave female professors significantly poorer ratings than they gave male professors on the six teaching evaluation measures; their ratings of female professors were poorer than those of female students on four of the six measures. Female students also evaluated female professors less favorably than male professors on three measures. Student perceptions of a professor's instrumental/active and expressive/nurturant traits, which were positively related to student ratings of teaching, accounted for only a few of these gender-related effects. Student major and student class standing also played a role in the evaluation of professors. The importance of gender variables in teacher evaluation studies is discussed, and implications for future research are noted. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of student race and physical attractiveness on teachers' judgments of transgressions.

60 student teachers (STs) and 137 practicing teachers (PTs) rated the severity of classroom misbehaviors ascribed to either a Black or White attractive or unattractive child. Following student teaching, STs' ratings of Black children's transgressions significantly increased in severity, while ratings of White children's transgressions remained the same. PTs were affected by student attractiveness but not by race, with transgressions by attractive children of both races being rated more severely than transgressions by unattractive children. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity and generalizability of student ratings from between-classes and within-class data.

Examined the generalizability and validity of student (N?=?485) ratings by studying within-class and between-classes correlations of ratings with other variables for regular faculty teaching lecture courses as well as for graduate assistants teaching recitation sections. Results indicate that most ratings were highly generalizable but only some were related to learning and that certain aspects of both generalizability and validity varied with instructor's role and with level of data. The implications of these findings for the evaluation of teaching are discussed with reference to 2 alternative paradigms: construct validity and criterion development. (22 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Microtraining of forensic psychiatric patients for empathic counseling skills.

Assessed the usefulness of a 7?-hr microtraining package for peer counselors in a therapeutic community of forensic psychiatric patients. 10 female and 6 male 17–38 yr old volunteers who were patients in a minimum-security forensic ward were randomly assigned to (1) the skills group, in which they were trained in the cognitive correlate of empathy and counseling skills; or (b) the attention group, in which they viewed counseling films. Multiple modes of measurement—written (e.g., the Carkhuff Communication Index [CCI]), observer ratings (e.g., staff ratings, a feeling checklist), and oral (e.g., a critical-incidents audiotape)—were used to assess knowledge acquisition, counseling effectiveness, and generalization to personality and behavior functioning. In comparison with the attention group, the skills group showed significantly greater written knowledge acquisition on the CCI and significantly greater use of counseling skills in oral responses to critical-incidents audiotapes and in counseling interviews. Effects did not generalize to a personality measure or to ward behavior, nor did increased use of counseling skills affect helpee self-disclosure in peer interviews. Longer training periods may be needed to effect broader changes. (26 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)