Accuracy of performance ratings as affected by rater training and perceived purpose of rating.

Investigated the effects of perceived purpose for rating and training type on the following dependent variables: accuracy, leniency/severity, and illusory halo. The purpose factor comprised 3 levels: a hiring purpose, a feedback purpose, and a research-only purpose. The training factor comprised 4 levels: rater error (RE) training, frame-of-reference (FOR) training, the combination of both methods, and no training. With both factors crossed, 164 undergraduates were randomly assigned to 1 of 12 conditions and viewed videotapes of lectures given by bogus graduate assistants. Heterogeneity of variance made it necessary to apply a conservative analytical strategy. Training significantly affected 2 measures of accuracy and halo such that a training condition that contained an FOR component did better than RE or no training. The conservativeness of the conservative analytic strategy made effects for the purpose factor on correlation accuracy, leniency/severity, and halo only tentative; it dissipated the 1 interaction effect of the 2 factors on distance accuracy. Discussion centers on (a) comparison of the results with those of S. Zedeck and W. Cascio (see record 1983-09102-001), (b) potential reasons for the heteroscedasticity, and (c) implications for the development of student evaluations of university instructors. (32 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Role of the rater in performance appraisal.

Notes that ratings for performance appraisal are frequently made by supervisors. In the present study, judgments of effectiveness for 153 hospital nurses were obtained from the nurse herself and her peers in addition to her supervisor, using the same rating form. Factor analysis indicated that each rating source could be clearly identified and characterized. The data reaffirm the notion that interrater disagreement may reflect systematic rater bias as well as meaningful differences in the ways in which judgments are made. Implications for comprehensive appraisals are suggested. (29 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A comparison of rater training programs: Error training and accuracy training.

108 undergraduates were randomly assigned to 1 of 4 experimental groups to rate videotaped performances of several managers talking with a problem subordinate. The research employed a single-factor experimental design in which rater error training (RET), rater accuracy training (RAT), rating error and accuracy training (RET/RAT), and no training were compared for 2 rating errors (halo and leniency) and accuracy of performance evaluations. Differences in program effectiveness for various performance dimensions were also assessed. Results show that RAT yielded the most accurate ratings and no-training the least accurate ratings. The presence of error training (RET or RET/RAT) was associated with reduced halo, but the presence of accuracy training (RAT or RET/RAT) was associated with less leniency. Dimensions?×?Training interactions revealed that training was not uniformly effective across the rating dimensions. (23 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effect of rater training on rater accuracy: Levels-of-processing theory and social facilitation theory perspectives.

We use levels-of-processing theory and social facilitation theory to explain the effect of training format and group size on distance and correlation accuracy, leniency-severity, halo, retention of training and pretraining information, and subject arousal. The training factor included frame-of-reference (FOR) training, information only (INFO) training, and no training (NOT). Group size was n?=?1, n?=?6, and n?=?12, respectively. A total of 108 subjects, randomly assigned to one of nine Training?×?Group Size conditions, viewed and rated videotaped lectures. Results indicated that FOR training effected improved retention of training information, improved distance accuracy, and less halo over INFO training or NOT (p     

Work attitudes and decisions as a function of manager age and employee age.

Research has shown the importance of employee age relative to coworker age in determining attitudes, performance, and career-related opportunities. The authors used chronological and subjective measures of employee and manager age to determine whether employee age relative to the manager has an impact on these same outcome variables. One hundred eighty-five managers and 290 employees completed surveys. The strongest and most consistent age effects were observed for interactions between employee and manager chronological age. Both the magnitude and pattern of the employee-manager age interactions varied by self- and manager-rated outcome measures of work attitudes, performance and promotability assessments, and developmental experiences. Results are discussed in light of the relational demography and career timetable literatures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of rater training and diary-keeping on psychometric error in ratings.

Compared the psychometric properties of ratings on behavioral expectation scales (BES) across 4 groups totalling 156 undergraduate raters. Groups differed with respect to amount of prior training (1 hr or more), the nature of psychometric errors, and the extent of exposure to scales (read scales and recorded observed critical incidents, discussed general scale dimensions, or no exposure to scales). Three Ss from each group rated 1 of 13 instructors during the last week of a 10-wk term. Significantly less leniency error and halo effect, plus higher interrater reliability, were found for the group that had received the hour of training and full exposure to the BES. Ss who had received only training had significantly less halo error than those that had received no training. The need for rater training prior to observation and the use of BES as a context for observation are discussed. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of purpose of the appraisal and expectation of validation on self-appraisal leniency.

Tested the hypothesis that (1) self-appraisals conducted under a grading purpose are more lenient than those conducted under a research purpose and (2) self-appraisals obtained under conditions with high expectation of validation are less lenient than those obtained under conditions with low expectation of validation with 62 undergraduates. Ss completed self-reports of the classroom participation; the questionnaires varied in the purpose of appraisal and the expectation of validation. Both hypotheses were strongly supported. (8 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Discrimination reversal in the goldfish as a function of training conditions.

Performed 3 experiments on goldfish (n = 11, 24, and 10, respectively) and found that Ss showed progressive improvement in red-green discrimination reversal under a variety of training conditions, the amount of improvement varying with conditions. Exposing the animals to S+ during reinforcement had the greatest facilitating effect on performance of the several modifications introduced. The use of a center key, response to which was required for presentation of the discriminanda, also produced substantial facilitation. However, feeding Ss at the locus of response was of relatively minor importance, and increasing the duration of unreinforced exposure to S- had no effect. Some questions about the relation between improvement in goldfish and improvement in more advanced animals are considered. (20 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Performance as a function of increased minority hiring.

Examined the impact of increasing the percentage of hires (above the T. A. Cleary [1968] fair-hiring level) from minority groups with demonstrated lower average job performance. Increased minority hiring resulted in only a small performance loss, even when minority hiring exceeded the minority applicant representation. However, when minorities were hired at a rate equal to or greater than their applicant representation, the expected performance loss among the hired minorities was much greater than performance loss across all hires. More important, the discrepancy in performance between majority and minority hires increased as minority hiring increased. With minority hiring above the level of minority applicant representation, this discrepancy exceeded the population difference in performance between the 2 groups. These findings suggest additional considerations when raising minority hiring above minority applicant representation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating frame-of-reference rater training effectiveness using performance schema accuracy.

Frame-of-reference training has been shown to be an effective intervention for improving the accuracy of performance ratings (e.g., Woehr & Huffcutt, 1994). Despite evidence in support of the effectiveness of frame-of-reference training, few studies have empirically addressed the ultimate goal of such training, which is to teach raters to share a common conceptualization of performance (Athey & McIntyre, 1987; Woehr, 1994). The present study tested the hypothesis that, following training, frame-of-reference–trained raters would possess schemas of performance that are more similar to a referent schema, as compared with control-trained raters. Schema accuracy was also hypothesized to be positively related to rating accuracy. Results supported these hypotheses. Implications for frame-of-reference training research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Performance impairment as a function of nitrogen narcosis.

The effects of nitrogen narcosis on the performance of several tasks was studied. 10 Ss were trained to a constant level of performance in a choice reaction time test, a motor coordination test, and a reasoning test. The amount of impairment was determined as a function of increased partial pressure of nitrogen, equivalent to 100 feet of sea water. The results indicated (a) significant decrease in performance for all Ss on all tests when compared with their individual sea level efficiencies, (b) a position relationship between degree of impairment and the complexity of the task; and (c) an initial loss in efficiency as pressure increased, with this level of impairment remaining relatively constant with increased duration of exposure. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Increase in assertiveness and actualization as a function of didactic training.

Investigated whether didactic training could increase level of assertiveness and actualization. 65 low assertive undergraduates were randomly assigned to 1 of 5 groups. All groups except a no-treatment control group were exposed to 3 20-min videotapes during 1 wk. Group 1 was exposed to the didactic principles of Rogerian theory; Group 2 was given the didactic principles of assertiveness theory. All Ss were given a test battery administered before treatment, after treatment, and again at a 3-wk follow-up. Both Rogerian and assertiveness training increased the level of actualization and the level of time competence at posttesting and at follow-up; the effect of didactic training was significantly greater than the increase caused by inspiration or expectation. Didactic training, however, did not affect assertiveness. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Performance appraisal and psychological distance.

This study investigated the rating attitudes of supervisors and subordinates and their reactions during public- and private-performance evaluations. 84 women nursing administrators were assigned roles as supervisors and subordinates and given instructions for subordinate-evaluation interviews. 24 interviews were conducted, 6 in private and 18 in public, with Os randomly selected from the nursing administrators. The results clearly indicated that supervisors were more negative in their initial subordinate-appraisal ratings than subordinates. In addition, Os were more negative in their ratings than participants. When evaluative interviews were conducted publicly, the subordinates experienced a number of negative reactions that were not evident in private interviews. The concepts of psychological distance and role stereotypes were discussed in explaining these results. (15 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence of nonverbal communication and rater proximity on impressions and decisions in simulated employment interviews.

Examined the influences of nonverbal communication and different methods of assessing a job applicant. 72 female Ss saw either an immediate or nonimmediate applicant while serving as an interviewer, observer, or television observer. (Immediacy was manipulated through several nonverbal channels of communication: eye contact, smiling, posture, interpersonal distance, and body orientation.) Both observers were yoked to the interviewer. Results clearly indicate that nonverbal communication had a significant effect on interview impressions and subsequent decisions. Further, several self-report measures suggest different degrees of involvement as a function of the rater's proximity. Implications for future research and novel interviewing techniques are discussed. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Longitudinal study of the effects of rater training on psychometric error in ratings.

66 supervisory engineers were randomly assigned to (a) an intense training group, (b) a discussion group, or (c) a nontrained comparison group. The intense training and discussion Ss received 14 hrs of rater training designed to minimize halo and leniency error and to use the organization's behavioral expectation scale for engineers more effectively. A longitudinal research design was used to study the halo and leniency errors 6 mo before training (TB), 6 mo after training (T?), and 12 mo after training (T??). Using covariance analysis, ANOVA with repeated measures, and planned comparisons, findings indicate that the intense training module (which included a 6-hr videotape block) was superior to the discussion and comparison groups in reducing halo and leniency error. However, a noticeable dissipation of training effect on these psychometric errors was identified when the T? and T?? data were examined. (12 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy.

Frame-of-reference (FOR) rater training is one technique used to impart a theory of work performance to raters. In this study, the authors explored how raters' implicit performance theories may differ from a normative performance theory taught during training. The authors examined how raters' level and type of idiosyncrasy predicts their rating accuracy and found that rater idiosyncrasy negatively predicts rating accuracy. Moreover, although FOR training may improve rating accuracy even for trainees with lower performance theory idiosyncrasy, it may be more effective in improving errors of omission than commission. The discussion focuses on the roles of idiosyncrasy in FOR training and the implications of this research for future FOR research and practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Performance as a function of motive strength and expectancy of goal-attainment.

"Particular motives (Achievement, Affiliation, Power, etc.) are conceived as latent dispositions to strive for certain goal states (aims) that are engaged in performance of an act when the cues of the situation arouse the expectancy that performance of an act is instrumental to attainment of the goal of the motive. The total motivation to perform the act is conceived as a summation of strengths of all the motives that have been aroused by appropriate expectancies of goal-attainment cued-off by the situation. The relationship of achievement motive (as measured by imaginative TAT stories) to performance is shown to be significantly positive when the expectancy that performance is instrumental to producing a feeling of pride in accomplishment is aroused and few if any other expectancies of goal-attainment are aroused." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Transfer of team skills as a function of type of training.

5 groups varying in training context (team vs. individual) and skill acquisition (individual, coordination, and communication skills) were compared at transfer on team (coordination of interceptions) and individual (number of interceptions) performance of a simulated radar-controlled aerial intercept task. Individual performance was unaffected by the training variables, but team performance was a positive function of the emphasis on coordination skills during training. When acquisition of coordination skills was held constant, context had no effect on transfer performance. Intrateam communications retarded performance but prohibiting these communications during training did not lessen their disruptive effect at transfer. This inhibitory influence of team communications reflected the verbal transmittal of information irrelevant to the task or more readily obtainable from the radar scopes. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of frame-of-reference training on rater accuracy under alternative time delays.

Investigated the effects of frame-of-reference (FOR) training on various indexes of distance and correlational accuracy under alternative time delays. 150 Ss were assigned randomly to either FOR- or control- (i.e., minimal) training conditions, with 1 of 3 time delays: (1) no delay between training, observation, and rating; (2) ratings performed 2 days following training and ratee observations; or (3) ratee observations and ratings completed 2 days following training. Hypotheses were proposed predicting specific relationships between accuracy, recall memory, and learning, depending on the delay period. Overall, results support the categorization perspective on FOR-training effectiveness; however, different results were obtained depending on the type of accuracy index and time delay. The implications of these findings are discussed in terms of how they relate to the conceptual distinction between distance and correlational accuracy and to the role of on-line, memory-based, and inference-memory-based processing in the ratings of FOR trained raters. (PsycINFO Database Record (c) 2010 APA, all rights reserved)