Understanding performance ratings: Dynamic performance, attributions, and rating purpose.

The present two studies integrate and extend the literatures on dynamic performance, performance attributions, and rating purpose, making several important contributions. First, examining attributions of dynamic performance, Study 1 predicted that performance mean and trend would affect judged ratee ability and effort and that performance variation would affect locus of causality; both predictions were supported by the results. Second, investigating the interaction between dynamic performance and rating purpose, Study 2 predicted that performance mean would have a stronger impact on administrative than on developmental ratings, whereas performance trend and variation would have a stronger impact on developmental than on administrative ratings; again, both predictions were borne out by the results. Third, both studies found that performance trend interacted with performance mean and variability to predict overall ratings. Fourth, both studies replicated main effects of dynamic performance characteristics on ratings in a different culture and, in Study 2, a sample of more experienced managers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Adaptability measures as predictors of performance ratings.

How well do measures of temperamental and motivational characteristics in training predict success of performance of officers as indicated by Officer Effectiveness Ratings? S were selected from 666 cadets at one base and 69 at another. Variables used were 3 global indicies of adaptability and 10 variables representing the common factors. "(a) measures of adaptability to training and Air Force Life are more highly related to later officer performance than are measures of aptitude or ability; (b) assessments of a man's functioning involving personal judgments of peers, superiors, and experts are predictive of later performance as an Air Force officer." (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty.

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The interactive effect of leader-member exchange and communication frequency on performance ratings.

The authors tested the hypothesis that communication frequency moderates the relationship between leader-member exchange (LMX) and job-performance ratings. In a study of 188 private sector workers, they found that LMX was more strongly related to job-performance ratings among individuals reporting frequent communication with the supervisor than among those reporting infrequent communication. At high levels of LMX, workers reporting frequent communication with the supervisor received more favorable job-performance ratings than did workers reporting infrequent communication. In contrast, at low levels of LMX, workers reporting frequent communication with the supervisor received less favorable job-performance ratings than workers reporting infrequent communication. The authors conducted a 2nd study of 153 public sector workers to provide a constructive replication and found similar results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach.

A review of research on job performance suggests 3 broad components: task, citizenship, and counterproductive performance. This study examined the relative importance of each component to ratings of overall performance by using an experimental policy-capturing design. Managers in 5 jobs read hypothetical profiles describing employees' task, citizenship, and counterproductive performance and provided global ratings of performance. Within-subjects regression analyses indicated that the weights given to the 3 performance components varied across raters. Hierarchical cluster analyses indicated that raters' policies could be grouped into 3 homogeneous clusters: (a) task performance weighted highest, (b) counterproductive performance weighted highest, and (c) equal and large weights given to task and counterproductive performance. Hierarchical linear modeling indicated that demographic variables were not related to raters' weights. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The five-factor model of personality and managerial performance: Validity gains through the use of 360 degree performance ratings.

This study investigated the usefulness of the five-factor model (FFM) of personality in predicting two aspects of managerial performance (task vs. contextual) assessed by utilizing the 360 degree performance rating system. The authors speculated that one reason for the low validity of the FFM might be the failure of single-source (e.g., supervisor) ratings to comprehensively capture the construct of managerial performance. The operational validity of personality was found to increase substantially (50%–74%) across all of the FFM personality traits when both peer and subordinate ratings were added to supervisor ratings according to the multitrait–multimethod approach. Furthermore, the authors responded to the recent calls to validate tests via a multivariate (e.g., multitrait–multimethod) approach by decomposing overall managerial performance into task and contextual performance criteria and by using multiple rating perspectives (sources). Overall, this study contributes to the evidence that personality may be even more useful in predicting managerial performance if the performance criteria are less deficient. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sociometric ratings as predictors of academic performance.

An experiment to determine the degree of accuracy with which sociometric ratings can be used to predict academic performance. A 14-item sociometric questionnaire was administered to 86 students before any academic grades were available. 2 rating measures, one emphasizing future job performance and the other dealing with personal adjustment, were found to be significantly related to academic performance (r = .40, p     

Presentation modality and indirect performance information: Effects on ratings, reactions, and memory.

The authors investigated whether raters integrate indirect (second-hand) information from an employee's co-worker with their direct observations when completing performance evaluations. Performance levels of direct and indirect information and presentation modality (auditory vs. textual) were manipulated (N=220). Results showed that indirect information was perceived to be of highest utility when the performance levels of the direct and indirect information were consistent. Confidence in performance ratings was lowest when the indirect source delivered negative performance feedback that was contrary to the rater's own positive observations. Indirect information was only reflected in the performance ratings when direct observations were positive. There was a significant 3-way interaction between performance level of the direct information, performance level of the indirect information, and presentation modality on memory for performance incidents. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluating frame-of-reference rater training effectiveness using performance schema accuracy.

Frame-of-reference training has been shown to be an effective intervention for improving the accuracy of performance ratings (e.g., Woehr & Huffcutt, 1994). Despite evidence in support of the effectiveness of frame-of-reference training, few studies have empirically addressed the ultimate goal of such training, which is to teach raters to share a common conceptualization of performance (Athey & McIntyre, 1987; Woehr, 1994). The present study tested the hypothesis that, following training, frame-of-reference–trained raters would possess schemas of performance that are more similar to a referent schema, as compared with control-trained raters. Schema accuracy was also hypothesized to be positively related to rating accuracy. Results supported these hypotheses. Implications for frame-of-reference training research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Improving peer supervisor ratings of therapist performance in dialectical behavior therapy: An Internet-based training system.

The present study involved the development of an Internet-based training system (ITS) to help train peer supervisors. The system, which was piloted with Dialectical Behavior Therapy (Linehan, 1993) using mock sessions, demonstrates how Internet-based technology can facilitate training protocols to support the ongoing training and supervision of therapists efficiently. Participant evaluation of the system was very positive, with over 90% of respondents reporting that they believed that the ITS was very useful or extremely useful for therapist training. Possible uses of the system include: (a) helping therapists to learn to discriminate more effective from less effective interventions to provide better feedback to supervisees and peers on their sessions, (b) helping therapists to improve their own ability to monitor and deliver a treatment effectively, and (c) helping to structure therapist training and supervision activities. The system could also be used to help with real-world supervision of therapists, assuming that legal and ethical issues associated with the use of the Internet for clinical supervision are addressed adequately. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding phylogenies in biology: The influence of a Gestalt perceptual principle.

Cladograms, hierarchical diagrams depicting evolutionary histories among (groups of) species, are commonly drawn in 2 informationally equivalent formats--tree and ladder. The authors hypothesize that these formats are not computationally equivalent because the Gestalt principle of good continuation obscures the hierarchical structure of ladders. Experimental results confirmed that university students (N = 44) prefer to subdivide ladders in accordance with good continuation rather than with the underlying hierarchical structure. Two subsequent experiments (N = 164) investigated cladogram understanding by examining students' ability to translate between formats (e.g., from tree to ladder). As predicted, students had greater difficulty understanding ladders than trees. This effect was larger for students with weaker backgrounds in biology. These results have important implications for evolution education reform. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The roles of rater goals and ratee performance levels in the distortion of performance ratings.

The goal-directed perspective of performance appraisal suggests that raters with different goals will give different ratings. Considering the performance level as an important contextual factor, we conducted 2 studies in a peer rating context and in a nonpeer rating context and found that raters do use different rating tactics to achieve specific goals. Raters inflated their peer ratings under the harmony, fairness, and motivating goal conditions (Study 1, N = 103). More important, raters inflated their ratings more for low performers than for high and medium performers. In a nonpeer rating context, raters deflated ratings for high performers to achieve the fairness goal, and they inflated ratings for low performers to motivate them (Study 2, N = 120). (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The moderating role of subordinate political skill on supervisors' impressions of subordinate ingratiation and ratings of subordinate interpersonal facilitation.

Nearly 2 decades ago, social influence theorists called for a new stream of research that would investigate why and how influence tactics are effective. The present study proposed that political skill affects the style of execution of influence attempts. It utilized balance theory to explain the moderating effect of employee political skill on the relationships between self- and supervisor-reported ingratiation. Additionally, supervisor reports of subordinate ingratiation were hypothesized to be negatively related to supervisor ratings of subordinate interpersonal facilitation. Results from a combined sample of 2 retail service organizations provided evidence that subordinates with high political skill were less likely than those low in political skill to have their demonstrated ingratiation behavior perceived by targets as a manipulative influence attempt. Also, when subordinates were perceived by their supervisors to engage in more ingratiation behavior, the subordinates were rated lower on interpersonal facilitation. Implications of these findings, limitations, and future research directions are provided. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Does a structured free recall intervention reduce the effect of stereotypes on performance ratings and by what cognitive mechanism?

The purpose of this article was to extend previous work on the effect of racial biases on performance ratings. The 1st of 2 studies examined whether a structured free recall intervention decreased the influence of negative racial biases on the performance ratings of Black men. Results indicated that without the intervention, raters who endorsed a negative stereotype of Black men as managers evaluated Black men more negatively. However, the structured free recall intervention successfully reduced these effects. The second study examined in more detail the cognitive mechanisms underlying the success of the intervention. Results are consistent with the assumption that the reduction of the influence of racial biases under structured free recall conditions is a consequence of a modified strength threshold for retrieval of behaviors from memory. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evidence of the construct validity of developmental ratings of managerial performance.

The construct validity of developmental ratings of managerial performance was assessed by using 2 data sets, each based on a different 360° rating instrument. Specifically, the authors investigated the nature of the constructs measured by developmental ratings, the structural relationships among those constructs, and the generalizability of results across 4 rater perspectives (boss, peer, subordinate, and self). A structure with 4 lower order factors (Technical Skills, Administrative Skills, Human Skills, and Citizenship Behaviors) and 2 higher order factors (Task Performance and Contextual Performance) was tested against competing models. Results consistently supported the lower order constructs, but the higher order structure was problematic, indicating that the structure of ratings is not yet well understood. Multisample analyses indicated few practically significant differences in factor structures across perspectives. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Bender Gestalt: A review and a perspective.

The 1950-61 literature is covered and is organized around the test designs, the various scoring systems, and the diagnostic interpretations. When scored objectively its validity in determining mental ages for children, and as an additional tool in a test battery aimed at differential diagnoses, is acceptable. Symbolic interpretation remains highly subjective. The lack of standardization mitigates against utilizing the test as a norm against which to judge other variables. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of observer ratings of the five-factor model of personality traits: A meta-analysis.

Conclusions reached in previous research about the magnitude and nature of personality–performance linkages have been based almost exclusively on self-report measures of personality. The purpose of this study is to address this void in the literature by conducting a meta-analysis of the relationship between observer ratings of the five-factor model (FFM) personality traits and overall job performance. Our results show that the operational validities of FFM traits based on observer ratings are higher than those based on self-report ratings. In addition, the results show that when based on observer ratings, all FFM traits are significant predictors of overall performance. Further, observer ratings of FFM traits show meaningful incremental validity over self-reports of corresponding FFM traits in predicting overall performance, but the reverse is not true. We conclude that the validity of FFM traits in predicting overall performance is higher than previously believed, and our results underscore the importance of disentangling the validity of personality traits from the method of measurement of the traits. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The specific effects of a Gestalt intervention.

Reports 3 cases in which the effects on client depth of experiencing and voice quality of the Gestalt "2-chair" operation were investigated. In a design using the clients as their own controls, the effects on each client of 3 Gestalt operations, applied under experimental control to client statements of conflict, were compared with the effects of 3 active-empathy operations at similar points. Results confirm that following the Gestalt operation the depth of experiencing was significantly higher for each of the 3 clients selected as good prognosis clients for client-centered therapy. Two Ss used significantly more emotionally expressive voice after the Gestalt operation. The implications of the change in depth of experiencing on shifts in awareness and on the client's ability to cope with and resolve conflict are discussed. (22 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The implications of linking the dynamic performance and turnover literatures.

This article examines how the literatures of dynamic performance and the performance–turnover relationship inform each other. The nonrandom performance–turnover relationship suggests that dynamic performance studies may be biased by their elimination of participants who do not remain for the entire study period. The authors demonstrated that the performance slopes of those who leave an organization differ from the performance slopes of those who remain. This finding suggests that studies of the performance–turnover relationship need to consider employee performance trends when predicting turnover. Replicating and extending the research of D. A. Harrison, M. Virick, and S. William (1996), the authors found that performance changes from the previous month and performance trends measured over a longer time period explained variance in voluntary turnover beyond current performance. Finally, the authors showed that performance trends interacted with current performance in the prediction of voluntary turnover. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The Bender Gestalt Test: A scoring method for identifying brain damage.

Comparisons between groups of 20 brain damaged, 38 psychiatric, and 25 normal Ss as to the occurences of 31 signs in Bender Gestalt protocols yeilded 15 signs which discriminated between brain damaged and non-brain-damaged groups. Protocols of the original sample of Ss and of a 2nd sample of matched psychiatric and brain damaged Ss (21 in each sample) were scored by using the discrimination weight of the signs derived from the original sample. Results indicated that the scoring system significantly differentiated between brain damaged and non-brain-damaged groups in both the original and cross-validation samples. Characteristics and limitations of the system are discussed. This new scoring method shows promise for objectively identifying impairment associated with brain damage. (PsycINFO Database Record (c) 2010 APA, all rights reserved)