Comparisons of self- and superior ratings of managerial performance.

Self- and superior ratings of performance on 9 dimensions were obtained from an interorganizational sample of 102 managers by a mail questionnaire. It was found that self-ratings possessed less leniency, restriction of range, and halo error than did superior ratings. The ratings only partially met the requirements of convergent and discriminant validity. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rater and technique contamination in criterion ratings.

Rater contamination may occur when the same rater makes both prediction and criterion ratings; technique contamination may occur when both the prediction and criterion ratings are on the same form. 400 Army officers provided ratings on a graphic scale and two versions of a forced-choice scale. Each officer rated 20 of his fellow officers on the graphic scale, and eight days later rerated two of them on the graphic scale and one of the forced-choice scales. Intercorrelations for the same raters on the same (graphic) form were .69-.82. Inter-correlations for the same rater on different forms were .52-.57. For different raters, inter-correlations were below .3, for same or different forms. It is concluded that the rater is the principal source of contamination. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Discrepancy between self- and observer ratings of performance in social phobics.

28 Ss meeting Diagnostic and Statistical Manual of Mental Disorders-III-Revised (DSM-III-R) criteria for social phobia and without comorbid affective disorder and 33 nonclinical controls were asked to present a brief, impromptu speech to a small audience. Speakers themselves, as well as members of the audience, rated each speaker on a public speaking questionnaire that included both specific items (e.g., voice shook) and global items (e.g., appeared confident). For global items, no significant difference was indicated between the 2 groups on observers' ratings of public speaking performance. However, social phobics rated their own performance worse than did nonclinical controls, and there was a significantly greater discrepancy between self and other ratings for social phobics than controls. Fear of negative evaluation was the only significant predictor of the self–other discrepancy on global items. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of ratee task performance and interpersonal factors on supervisor and peer performance ratings.

The study examines the effects of a wide array of rater–ratee relationship and ratee-characteristic variables on supervisor and peer job-performance ratings. Interpersonal ratings, job performance ratings, and ratee scores on ability, job knowledge, and technical proficiency were available for 493 to 631 first-tour US Army soldiers. Results of supervisor and peer ratings-path models showed ratee ability, knowledge, and proficiency accounted for 13% of the variance in supervisor performance ratings and 7% for the peer ratings. Among the interpersonal variables, ratee dependability had the strongest effect for both models. Ratee friendliness and likability had little effect on the performance ratings. Inclusion of the interpersonal factors increased the variance accounted for in the ratings to 28% and 19%, respectively. Discussion focuses on the relative contribution of ratee technical and contextual performance to raters' judgments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Construct validity of self- and peer evaluations of performance dimensions in an assessment center.

The construct validity of final self- and peer evaluations in an assessment center was examined within a nomological network of conceptually related and unrelated variables. Data included self-, peer, and assessor evaluations; cognitive ability and personality measures; and job advancement. The evidence for construct validity was stronger for peer than for self-evaluations and for more easily observable dimensions than for dimensions requiring greater inferential judgment. Self- and peer evaluations were associated with assessor ratings of management potential, whereas only peer evaluations predicted job advancement. Implications for the use of self- and peer evaluations in assessment centers and the need for further research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rater bias in the EASI temperament scales: A twin study.

Under trait theory, ratings may be modeled as a function of the temperament of the child and the bias of the rater. Two linear structural equation models are described, one for mutual self and partner ratings, and one for multiple ratings of related individuals. Application of the first model to EASI temperament data collected from spouses rating each other shows moderate agreement between raters and little rating bias. Spouse pairs agree moderately when rating their twin children, but there is significant rater bias, with greater bias for monozygotic than for dizygotic twins. MLEs of heritability are approximately .5 for all temperament scales with no common environmental variance. Results are discussed with reference to trait validity, the person–situation debate, halo effects, and stereotyping. Questionnaire development using ratings on family members permits increased rater agreement and reduced rater bias. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Perceived similarity and accuracy of peer ratings.

The hypothesis that raters will be more accurate in rating peers perceived to be similar to themselves, suggested by Mumford (1983) and derived from social comparison theory, was examined. Subjects were 681 Israeli entrants to a military training program. Shortly after course inception, subjects were asked to review the performance of squad members and to forecast their final grade. Subjects also judged peers' similarity to self overall (general similarity), in course achievement (foreground similarity), and in military experience (background similarity). Analysis revealed that accuracy was markedly lower in the evaluation of dissimilar others. Although the same patterns of results was observed for all forms of similarity, stronger effects resulted when similarity was measured in terms of general and foreground characteristics. Implications for future theory and research as well as for the practical application of peer assessment are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding performance ratings: Dynamic performance, attributions, and rating purpose.

The present two studies integrate and extend the literatures on dynamic performance, performance attributions, and rating purpose, making several important contributions. First, examining attributions of dynamic performance, Study 1 predicted that performance mean and trend would affect judged ratee ability and effort and that performance variation would affect locus of causality; both predictions were supported by the results. Second, investigating the interaction between dynamic performance and rating purpose, Study 2 predicted that performance mean would have a stronger impact on administrative than on developmental ratings, whereas performance trend and variation would have a stronger impact on developmental than on administrative ratings; again, both predictions were borne out by the results. Third, both studies found that performance trend interacted with performance mean and variability to predict overall ratings. Fourth, both studies replicated main effects of dynamic performance characteristics on ratings in a different culture and, in Study 2, a sample of more experienced managers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improving peer supervisor ratings of therapist performance in dialectical behavior therapy: An Internet-based training system.

The present study involved the development of an Internet-based training system (ITS) to help train peer supervisors. The system, which was piloted with Dialectical Behavior Therapy (Linehan, 1993) using mock sessions, demonstrates how Internet-based technology can facilitate training protocols to support the ongoing training and supervision of therapists efficiently. Participant evaluation of the system was very positive, with over 90% of respondents reporting that they believed that the ITS was very useful or extremely useful for therapist training. Possible uses of the system include: (a) helping therapists to learn to discriminate more effective from less effective interventions to provide better feedback to supervisees and peers on their sessions, (b) helping therapists to improve their own ability to monitor and deliver a treatment effectively, and (c) helping to structure therapist training and supervision activities. The system could also be used to help with real-world supervision of therapists, assuming that legal and ethical issues associated with the use of the Internet for clinical supervision are addressed adequately. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An industrial use of peer ratings.

Can peer ratings identify managers who later are promoted rapidly in a large corporation? A representative sample of 56 graduates was selected from 4-week middle manager training classes where a lengthy 13 item peer rating was administered. When the mean peer ratings of 21 managers who received 2 or more promotions after graduation were compared with the mean peer ratings of 19 managers who received no promotions, differences were significant at the .01 level for all but 3 of the rating items. The results suggest that a large body of predictive data for industry exists in the judgments contemporary managers are able to make about each other. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of comparative performance information on the accuracy of self-ratings and agreement between self- and supervisor ratings.

The manner in which social comparison performance information affects the accuracy of self-ratings and the agreement between self- and supervisor ratings was investigated in a laboratory experiment using 163 undergraduates. Ss proofread a series of articles for 30 min and then made self-ratings of work performance. Half of the subjects received social comparison information prior to making self-ratings. Correlations between self- and supervisor evaluations and between self-evaluations and objective performance indicators were significantly larger when self-raters were presented with the same comparative performance information that was available to supervisors. Implications of the findings for future research on self- and supervisor evaluations are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Rater confidence and social influence in performance appraisals.

The influence of rater confidence on combined evaluations was examined when information from multiple sources was available. Confidence was related to evaluation extremity for novice raters when no other rater information was available. Information from multiple raters differentially affected the relationship between confidence and rating extremity. Evaluation information supplied by a highly confident, extreme rater significantly influenced evaluations from noninteracting novice raters but only minimally influenced evaluations from interacting novice raters. Among experts, raters seen as raising the greatest number of and the most persuasive arguments were most influential. The most accurate rater in a group had little effect on combined evaluations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Magnitude and moderators of bias in observer ratings: A meta-analysis.

Bias in observer ratings compromises generalizability of measurement, typically resulting in attenuation of observed associations between variables. This quantitative review of 79 generalizability studies including raters as a facet examines bias in observer ratings in published psychological research and identifies properties of rating systems likely to place them at risk for problems with rater bias. For the rating systems studied, an average of 37% of score variance was attributable to 2 types of rater bias: (a) raters' differential interpretations of the rating scale and (b) their differential evaluations of the same targets. Ratings of explicit attributes (e.g., frequency counts) contained negligible bias variance, whereas ratings of attributes requiring rater inference contained substantial bias variance. Rater training ameliorated but did not solve the problem of bias in inferential rating scales. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Revealing feelings: facets of emotional expressivity in self-reports, peer ratings, and behavior

Nuclear dots containing PML and Sp100 proteins (NDs) play a role in the development of acute promyelocytic leukemia, are modified after infection with various viruses, and are autoimmunogenic in patients with primary biliary cirrhosis (PBC). PML and Sp100 gene expression is strongly enhanced by interferons (IFN). Based on immunostaining with a monoclonal antibody (mAb C8A2), a third protein, nuclear dot protein 52 (NDP52), was recently localized in NDs. Here we analyzed the cellular localization, expression, and structure of NDP52 in more detail. Our NDP52-specific sera revealed mainly cytoplasmic staining but no ND pattern, neither in untreated nor in IFN-treated cells. Cells transfected with NDP52 expression vectors showed exclusively cytoplasmic staining. In subcellular fractionation experiments, NDP52 was found in cytoplasmic and nuclear fractions. Unlike as described for Sp100 and PML, NDP52 mRNA and protein levels were only marginally enhanced by IFN gamma and not enhanced at all by IFN beta. NDP52 homodimerization but no heterodimerization with Sp100 or PML could be demonstrated. None of the 93 PBC sera tested contained autoantibodies against NDP52. Finally, mAb C8A2 reacted not only with NDP52 but also with a conformation-dependent epitope on the Sp100 protein. These data imply that NDP52 forms homodimers but no heterodimers with Sp100 and PML, lacks autoantigenicity in PBC, localizes mainly in the cytoplasm, and is associated with the nucleus, but not with NDs. Finally, unlike Sp100 and PML, NDP52 expression is neither markedly enhanced nor localization detectably altered by type I and II IFNs.     

Prediction of managerial success by peer and training-staff ratings.

Examined the predictive validity of peer and training-staff ratings made under "real" conditions in an industrial situation. Ratings were obtained for 156 middle-level managers and 83 higher level executives attending a 4-wk training course. Analysis of 13 peer-related characteristics yielded 2 factors: Impact and Tactfulness. Follow-up showed that later success (promotion and performance appraisal) was predicted by most peer ratings mainly from the Impact factor. Training staff ratings were much weaker predictors and added little to the peer ratings. Broader use of peer ratings for executive selection is discussed. (15 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

"Rater bias in the EASI temperament scales: A twin study": Erratum.

Reports an error in the original article by M. C. Neale and J. Stevenson (Journal of Personality and Social Psychology, 1989, Vol 56[3], 446–455). Owing to a typographic error in computer programming, Tables 3 and 4 contain incorrect parameter estimates and chi-square values. The correction affects only the analysis of spouse and self-ratings, which was not the article's primary focus. Differences between original and revised statistics are outlined, and the corrected tables are presented. (For original abstract, see record 1989-17547-001). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparison of peer assessment methods: Reliability, validity, friendship bias, and user reaction.

Compared peer nominations, peer rankings, and peer ratings as to reliability, validity, friendship bias, and user reaction. 145 police officers provided each type of peer assessment. Criterion rankings and ratings were provided by 33 squad supervisors. All peer-based methods showed significant reliability and validity, and the validity coefficients were not significantly biased by the friendship between peer assessor and assessee. Peer rankings and peer nominations displayed significantly greater reliability and validity than did peer ratings. User reactions were negative for all methods. Recommendations are made for collection of peer assessment data in nonpolice organizations for comparative purposes and for investigation of techniques to reduce negative reactions to peer assessment. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty.

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effect of race on peer ratings in an industrial situation.

Examined the effect of race on the peer ratings of 43 black and 50 white industrial employees recently exposed to a foreman-training program which included intensive human relations training. Contrary to previous studies, no race effect was found. In addition, almost all the requirements for convergent and discriminant validity between the races were met. Possible explanations for these results and implications for the use of peer ratings in integrated settings are discussed. (15 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)