Rating leniency and halo in multisource feedback ratings: Testing cultural assumptions of power distance and individualism-collectivism.

This study extends multisource feedback research by assessing the effects of rater source and raters' cultural value orientations on rating bias (leniency and halo). Using a motivational perspective of performance appraisal, the authors posit that subordinate raters followed by peers will exhibit more rating bias than superiors. More important, given that multisource feedback systems were premised on low power distance and individualistic cultural assumptions, the authors expect raters' power distance and individualism-collectivism orientations to moderate the effects of rater source on rating bias. Hierarchical linear modeling on data collected from 1,447 superiors, peers, and subordinates who provided developmental feedback to 172 military officers show that (a) subordinates exhibit the most rating leniency, followed by peers and superiors; (b) subordinates demonstrate more halo than superiors and peers, whereas superiors and peers do not differ; (c) the effects of power distance on leniency and halo are strongest for subordinates than for peers and superiors; (d) the effects of collectivism on leniency were stronger for subordinates and peers than for superiors; effects on halo were stronger for subordinates than superiors, but these effects did not differ for subordinates and peers. The present findings highlight the role of raters' cultural values in multisource feedback ratings. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Effects of rater accountability on the accuracy and the favorability of performance ratings.

The authors tested the effects of holding raters accountable for their performance ratings on the accuracy and the favorability of those ratings. Undergraduate research participants (N?=?247) completed an inbasket exercise and observed a videotaped simulation during 2 sessions over a 2-wk period. The simulation presented performance information on 4 simulated subordinates portrayed through videotaped vignettes. True performance scores were manipulated by varying the proportion of positive and negative performance vignettes presented for each subordinate. Participants who were made to feel accountable by having to justify their ratings to the experimenter in writing rated their simulated subordinates more accurately. In another experimental condition, accountable raters who were told their subordinates' previous performance ratings were too low rated their subordinates more favorably than did raters in the same experimental condition who were not accountable. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparisons of self- and superior ratings of managerial performance.

Self- and superior ratings of performance on 9 dimensions were obtained from an interorganizational sample of 102 managers by a mail questionnaire. It was found that self-ratings possessed less leniency, restriction of range, and halo error than did superior ratings. The ratings only partially met the requirements of convergent and discriminant validity. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Superiors' evaluations and subordinates' perceptions of transformational and transactional leadership.

Transformational leaders are postulated to be responsible for performance beyond ordinary expectations as they transmit a sense of mission, stimulate learning experiences, and arouse new ways of thinking. Transactional leaders achieve performance as merely required by the use of contingent rewards or negative feedback. Previous research has shown that subordinates' perceptions of transformational leadership add to the prediction of subordinates' satisfaction and effectiveness ratings beyond that of perceptions of transactional leadership. The present study replicates the previous augmentation effects using subordinates' effectiveness ratings but was unable to confirm the augmentation hypothesis with independently attained superiors' evaluations as the criteria because of smaller sample size, although trends in the correlations were in the hypothesized direction. Transformational leadership obtained from their subordinates' ratings significantly differentiated top performing managers (identified as such through other sources) from ordinary managers as hypothesized. Results are discussed as they relate to a domestic work force that is becoming better educated and is more concerned about interesting work and self-development. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A comparison of rater training programs: Error training and accuracy training.

108 undergraduates were randomly assigned to 1 of 4 experimental groups to rate videotaped performances of several managers talking with a problem subordinate. The research employed a single-factor experimental design in which rater error training (RET), rater accuracy training (RAT), rating error and accuracy training (RET/RAT), and no training were compared for 2 rating errors (halo and leniency) and accuracy of performance evaluations. Differences in program effectiveness for various performance dimensions were also assessed. Results show that RAT yielded the most accurate ratings and no-training the least accurate ratings. The presence of error training (RET or RET/RAT) was associated with reduced halo, but the presence of accuracy training (RAT or RET/RAT) was associated with less leniency. Dimensions?×?Training interactions revealed that training was not uniformly effective across the rating dimensions. (23 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality moderators of the relationship between abusive supervision and subordinates' resistance.

Results of a study using data collected at 2 points in time, separated by 6 months, suggested that subordinates resisted their supervisors' downward influence tactics with greater frequency when their supervisors were more abusive and that subordinates' personality moderated the effects of abusive supervision. The relationship between abusive supervision and subordinates' dysfunctional resistance was stronger among subordinates who were lower in conscientiousness than among subordinates who were higher in conscientiousness, but this effect emerged only for subordinates who were also lower in agreeableness. The relationship between abusive supervision and subordinates' constructive resistance was stronger among subordinates who were higher in conscientiousness than among subordinates who were lower in conscientiousness. The study's implications for theory and research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sex effects on performance ratings on manager–subordinate dyads: A field study.

Tested the hypothesis of N. Schmitt et al (see record 1980-33528-001) that people rate those similar to themselves with more confidence, which is reflected in larger variances in performance ratings. 286 manager–subordinate dyads in all 4 sex combinations comprised the sample. Subordinates rated managers on the Behavioral Observation Scales (BOS), and managers rated subordinates using the Minnesota Satisfactoriness Scales (MSS). Bartlett's test for homogeneity of variance revealed that female subordinates produced significantly more variability when rating their male managers than when rating their female managers on total BOS rating. Female managers produced more variable ratings of male subordinates than for female subordinates on total MSS rating. Male subordinates and managers did not produce more variable BOS or MSS ratings when appraising other males than when rating females. Results are discussed in terms of role expectations. (24 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Self-determination in a work organization.

Research testing self-determination theory was discussed in terms of recent work on intrinsic motivation, participative management, and leadership. On three occasions, managers' interpersonal orientations—toward supporting subordinates' self-determination versus controlling their behavior—were related to perceptions, affects, and satisfactions of the subordinates. Data from 23 managers and their subordinates in a major corporation showed that managers' orientations did correlate with the subordinate variables, although the magnitude of the relation varied, seemingly as a function of factors in the corporate climate. An organizational development intervention, focused on the concept of supporting subordinates' self-determination, was provided for the managers. Evaluation of the program showed a clearly positive impact on managers' orientations, though a less conclusive radiation to subordinates. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evidence incompatible with a cognitive compatibility theory of rating behavior.

Assessed the cognitive complexity of 96 undergraduates with the group version of the Role Construct Repertory (REP) Test, a factor analysis of REP test data, and a sorting task. Performance ratings for 3 of the Ss' instructors were obtained with behaviorally anchored rating scales, mixed standard rating scales, graphic rating scales, and simple "alternate" 3-point rating scales. No differences in leniency, halo, or range restriction emerged either as a function of raters' cognitive complexity or a Cognitive Complexity?×?Scale Format interaction. Raters' confidence in their ratings was not associated with either cognitive complexity or rating scale format. It is concluded that researchers of performance ratings should exercise restraint before confidently conferring moderator variable status on a cognitive complexity construct. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A comparison of behavioral expectation scales and graphic rating scales.

Compared ratings derived from behavioral expectation scales developed by 147 personnel management students with ratings based on graphic rating scales. The ratees were 4 college professors, and the raters were the 183 students in their classes. The behaviorally anchored scales resulted in less halo error, or alternatively, more independence in ratings of different dimensions of performance. The behaviorally anchored scales did not correct for leniency in ratings. These results were observed both among raters who participated in developing the behavioral expectation scales and among similar raters who did not take part in this process. The factor structures of the 2 rating formats were essentially equivalent in "cleanness." Neither solution was judged superior to the other. However, the behavioral expectation scale format possessed greater discriminant validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

When supervisors feel supported: Relationships with subordinates' perceived supervisor support, perceived organizational support, and performance.

The authors surveyed full-time retail employees and their supervisors to investigate relationships of supervisors' perceived organizational support (POS) with subordinates' perceptions of support from their supervisors (perceived supervisor support [PSS]), POS, and in-role and extra-role performance. The authors found that supervisors' POS was positively related to their subordinates' perceptions of supervisor support. Subordinates' PSS, in turn, was positively associated with their POS, in-role performance, and extra-role performance. Beyond these bivariate relationships, subordinates' perceptions of support from the supervisor mediated positive relationships of the supervisors' POS with the subordinates' POS and performance. These findings suggest that supervisors who feel supported by the organization reciprocate with more supportive treatment for subordinates. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The development and evaluation of behaviorally based rating scales.

Makes a distinction between criterion measures that assess individual performance in terms of concrete job functions and those that reflect organizational outcomes several steps removed from actual behavior (e.g., salary level). It is argued that psychologists should be trying to measure and predict the former, and a modification of the method of scaled expectations is suggested as 1 technique for doing so. The method was used to develop 9 criterion dimensions for department managers in a nationwide retail chain. The resulting behavior rating scales were compared with a summated ratings technique on a sample of 537 department managers. The behavioral scales yielded less method variance, less halo error, and less leniency error. Additional benefits from the method are noted. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Influence tactics, affect, and exchange quality in supervisor-subordinate interactions: A laboratory experiment and field study.

The hypothesis was tested that subordinates' impression-management tactics and performance affect supervisor–subordinate exchange quality by influencing supervisors' liking for and performance ratings of their subordinates. In Study 1, 96 undergraduates completed measures of liking, performance, and exchange quality after interacting with a subordinate who engaged in a high or low level of impression management and performed at a high, average, or low level of competence. LISREL results were consistent with the proposed model. In Study 2, 84 bank employees rated how frequently they engaged in each of 24 impression-management behaviors. Factor analysis revealed three types of tactics: job-focused, self-focused, and supervisor-focused. The employees' direct supervisors completed measures of liking, performance, and exchange quality. LISREL results indicated that supervisor-focused tactics affected supervisors' liking for subordinates, which in turn influenced exchange quality. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Well-connected leaders: The impact of leaders' social network ties on LMX and members' work attitudes.

We examined the proposition that leaders' social network ties in the larger organization influence the quality of their leader–member exchange (LMX) with their employees, which, in turn, impacts these employees' job satisfaction and turnover intentions. Using multilevel, multisource data from a field study of 184 bank employees nested within 42 branch managers, we found that leaders who had higher quality relationships with their bosses and who were more central in their peer networks were perceived by their subordinates as having greater status in the organization and, therefore, were able to form higher quality relationships with them. Further, the effects of the leaders' perceived status on LMX were stronger when subordinates were less central in their own peer network. Finally, LMX mediated the impact of leaders' perceived status in the organization on subordinates' job satisfaction and turnover intentions. Implications for theory and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rater bias in performance ratings: Superior, self-, and peer ratings.

Leniency errors, halo effects, and differential dimensionality were explored in an analysis of superior, self-, and peer performance ratings of 107 managerial and 76 professional employees in a medium-sized manufacturing location, representing 95% of the managerial and professional staff. Self-ratings showed greater leniency effects than superior or peer ratings. A multitrait–multimethod (MTMM) analysis supported the presence of strong halo effect and significant convergent validity but not discriminant validity. The dimensional analysis supported the presence of strong halo effect. A statistical control procedure for the halo effect was developed that involved calculating residuals of the performance items, controlling for the "overall effectiveness" variance component in each item. The procedure did not reduce the significant halo effect, nor did it improve the nonsignificant discriminant validity in the MTMM analysis. It did, however, clarify the dimensional structure of ratings by superiors. Data from 4 previously published studies were also reanalyzed using the statistical control procedure. (19 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of multisource feedback and a feedback facilitator on the influence behavior of managers toward subordinates.

The authors compared a feedback workshop with both a no-feedback control group and a comparison group of managers who received a feedback report but no feedback workshop. The multisource feedback was based on ratings of a manager's influence behavior by subordinates, peers, and bosses. Managers in the feedback workshop increased their use of some core influence tactics with subordinates, whereas there was no change in behavior for the control group or for the comparison group. The feedback was perceived to be more useful by managers who received it in a workshop with a facilitator than by managers who received only a printed feedback report. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of instructions to avoid halo error on reliability and validity of performance evaluation ratings.

Investigated the effects of a short training session designed to reduce halo error in performance ratings. 90 low and middle managers rated 1 of 6 hypothetical first-line supervisors on 6 performance dimensions according to behavior displayed in a prepared vignette. Ratings were taken prior to and following the 5-min training session, with rater-ratee combinations counterbalanced. The vignettes were developed to contain previously scaled behavior examples, thus enabling the calculation of "true" criterion scores for each dimension. Comparisons between these "true" criterion and the performance ratings revealed that the training session significantly reduced halo, while leaving validity of the ratings generally unaffected. Performance ratings completed after training possessed lower reliability, although raters provided somewhat more accurate performance profiles. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multitrait–multimethod–multirater analysis of performance ratings for law enforcement personnel.

144 deputy sheriffs were rated on 9 job performance dimensions with 2 rating scales by 2 raters. Results indicate that the rating scales (the Multiple Item Appraisal Form and the Global Dimension Appraisal Form) developed in this study were able to minimize the major problems often associated with performance ratings (i.e., leniency error, restriction of range, and low reliability). A multitrait/multimethod analysis indicated that the rating scales possessed high convergent and discriminant validity. A multitrait/multirater analysis indicated that although the interrater agreement and the degree of rated discrimination on different traits by different raters were good, there was a substantial rater bias, or strong halo effect. This halo effect in the ratings, however, may really be a legitimate general factor rather than an error. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Supervisors' role conflict and role ambiguity differential relations with performance ratings of subordinates and the moderating effect of screening ability.

Survey data obtained from two independent samples of supervisors (Ns?=?68 and 109) supported the prediction that supervisors' experience of role conflict would correlate positively with their reported tendency to deliberately inflate subordinates' performance ratings. Contrary to expectation, in neither sample did supervisors' screening ability emerge as a moderator of the relation between role conflict and rating inflation. In a third independent sample of supervisors (N?=?25), in which rating inflation was operationalized on the basis of actual performance ratings of subordinates, strong support was obtained for both the predicted positive association between role conflict and rating inflation as well as the predicted moderating effect of screening ability on this relation. By contrast, the authors did not anticipate that supervisors' experience of role ambiguity would be directly associated with, or interact with screening ability to predict, rating inflation. The data from all three samples were, in general, in line with these expectations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An empirical comparison of the relative effects of rater response biases on three rating scale formats.

Compared Smith-Kendall type behaviorally anchored scales for derived performance dimensions (Format 1), scales for the same dimensions but without the behavioral anchors (Format 2), and scales for dimensions selected on an a priori basis (Format 3) on the basis of susceptibility to rater response biases. Raters were 30 graduate students and ratees were 3 associate professors whom the raters had had in succession during their 1st year of graduate study. Leniency error and composite halo error were present in all ratings; there was no evidence of relative or absolute halo errors in any ratings. There was some evidence that the use of scales for derived dimensions reduced leniency error and increased the amount of variance attributable to ratee differences. The scale reliabilities of the 3 formats were also determined. A discussion of the feasibility of obtaining relatively independent scales for several job performance dimensions is included. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)