Variability across sources of performance ratings.

Examined the comparability of peer ratings, supervisor ratings, and self-ratings of counseling performance early in training. Earlier studies of the relationship of performance ratings from different sources have indicated some comparability across source of rating, particularly late in the training process. The results of the present study with 30 21–50 yr old graduate students enrolled in a prepracticum counseling methods course indicate considerable variability across sources of ratings used early in training. Implications and practical training strategies are discussed. (9 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of ratee task performance and interpersonal factors on supervisor and peer performance ratings.

The study examines the effects of a wide array of rater–ratee relationship and ratee-characteristic variables on supervisor and peer job-performance ratings. Interpersonal ratings, job performance ratings, and ratee scores on ability, job knowledge, and technical proficiency were available for 493 to 631 first-tour US Army soldiers. Results of supervisor and peer ratings-path models showed ratee ability, knowledge, and proficiency accounted for 13% of the variance in supervisor performance ratings and 7% for the peer ratings. Among the interpersonal variables, ratee dependability had the strongest effect for both models. Ratee friendliness and likability had little effect on the performance ratings. Inclusion of the interpersonal factors increased the variance accounted for in the ratings to 28% and 19%, respectively. Discussion focuses on the relative contribution of ratee technical and contextual performance to raters' judgments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Three approaches to the investigation of subgroup bias in performance measurement: Review, results, and conclusions.

Three methods of assessing subgroup bias in performance measurement commonly found in the literature are identified. After a review of these approaches, findings are reported from analyses of data collected in the US Army's Project A (J. P. Campbell, 1987). Correlations between nonrating performance measures and supervisor ratings were generally not moderated by race, but correlations between nonrating indicators of negative performance and ratings assigned by peers were. In addition, significant interactions between rater and ratee race on performance ratings were not eliminated when variance in the nonrating measures was removed from the ratings provided by Black and White raters. Conclusions about the magnitude and nature of bias in supervisor and peer ratings are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

An assessment of the gap between counseling theory and practice.

Used microcounseling procedures to assess differences between intended counseling behavior and actual practice among 43 graduate students in counselor education. A paradigm developed by D. H. Frey was used to divide counseling behavior into 4 types: rational-insight counseling, affective-insight counseling, affective-action counseling, and rational-action counseling. Ss met with 1 of 2 distinctly different coached clients in a 4-min videotaped microcounseling interview in which both self- and peer ratings were obtained. No significant differences were found between intended and actual behavior as perceived by the counselors themselves, but their intended behavior was significantly different from peer perceptions of interview behavior. Client type had no effect. Results indicate that beginning counselors have a need to see their counseling as being theoretically consistent, even when peer ratings indicate a gap between the beginning theoretical stance and actual practice. (32 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Describing and facilitating effective supervision behavior in counseling trainees.

Counseling trainees often do not receive formal assistance in assuming the role of supervisee. What constitutes effective trainee behavior within the context of supervision, as opposed to the clinical context, also has received little empirical attention. A national sample of 176 participants (145 supervisees, 31 supervisors) affiliated with counseling psychology or counseling center internship programs rated the importance of 52 behaviors/characteristics (Supervision Utilization Rating Form; SURF) to the effective use of supervision at specific developmental training levels. Supervisors and supervisees applied ratings from important to extremely important to all items. Statistically significant differences between supervisor and supervisee ratings were noted on 6 items. The development of the SURF and its potential uses in supervisee role induction are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Can a manager have a life and a career? International and multisource perspectives on work-life balance and career advancement potential.

The present study was the first cross-national examination of whether managers who were perceived to be high in work-life balance were expected to be more or less likely to advance in their careers than were less balanced, more work-focused managers. Using self ratings, peer ratings, and supervisor ratings of 9,627 managers in 33 countries, the authors examined within-source and multisource relationships with multilevel analyses. The authors generally found that managers who were rated higher in work-life balance were rated higher in career advancement potential than were managers who were rated lower in work-life balance. However, national gender egalitarianism, measured with Project GLOBE scores, moderated relationships based on supervisor and self ratings, with stronger positive relationships in low egalitarian cultures. The authors also found 3-way interactions of work-life balance ratings, ratee gender, and gender egalitarianism in multisource analyses in which self balance ratings predicted supervisor and peer ratings of advancement potential. Work-life balance ratings were positively related to advancement potential ratings for women in high egalitarian cultures and men in low gender egalitarian cultures, but relationships were nonsignificant for men in high egalitarian cultures and women in low egalitarian cultures. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty.

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of observer ratings of the big five personality factors.

Examined the validity of observer (supervisor, coworker, and customer) ratings and self-ratings of personality measures. Results based on a sample of 105 sales representatives supported the 2 hypotheses tested. First, supervisor, coworker, and customer ratings of the 2 job-relevant personality dimensions—conscientiousness and extraversion—were valid predictors of performance ratings, and the magnitude of the validities was at least as large as for self-ratings. Second, supervisor, coworker, and customer ratings accounted for significant variance in the criterion measure beyond self-ratings alone for the relevant dimensions. Overall, the results suggest that validities of personality measures based on self-assessments alone may underestimate the true validity of personality constructs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

White Racial Identity Dyadic Interactions in Supervision: Implications for Supervisees' Multicultural Counseling Competence.

Examining supervisory dyads consisting of a White supervisor and a White supervisee, the authors sought to determine the effects of similarities and differences in levels of supervisor and supervisee racial identity schemas or attitudes on White supervisees' self-reported multicultural counseling competence and multicultural case conceptualization ability. White supervisees in supervision dyads characterized by more advanced White racial identity schemas reported higher self-perceived multicultural counseling competence and obtained higher multicultural case conceptualization ratings than did their counterparts in supervision dyads characterized by lower White racial identity schemas. Implications of the findings are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relation between types of aggression and sociometric status: Peer and teacher perceptions.

Examined the relation between aggressive behavior and sociometric status (the extent to which a child is liked by his or her peers) in third and fourth graders. 47 boys and 51 girls from a northern New England elementary school were given a group-administered peer rating of social status and a peer nomination measure assessing five types of aggressive behavior. Teachers of the same children were given modified versions of these measures. Significant negative correlations were found between social status and all categories of aggressive behavior for both sexes except provoked physical aggression in boys for which the correlation was not significant. Indirect aggression (tattling, stealing, or breaking others' property) was the type of aggressive behavior that correlated most highly with low peer ratings. There were significant differences between boys and girls on all five categories of aggressive behavior. Teachers' ratings of peer social status correlated more highly with boys' ratings than with girls' ratings, and teacher perceptions of aggressive behavior correlated significantly and positively with peers' ratings on only two categories, unprovoked physical aggression and indirect aggression. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Microtraining of forensic psychiatric patients for empathic counseling skills.

Assessed the usefulness of a 7?-hr microtraining package for peer counselors in a therapeutic community of forensic psychiatric patients. 10 female and 6 male 17–38 yr old volunteers who were patients in a minimum-security forensic ward were randomly assigned to (1) the skills group, in which they were trained in the cognitive correlate of empathy and counseling skills; or (b) the attention group, in which they viewed counseling films. Multiple modes of measurement—written (e.g., the Carkhuff Communication Index [CCI]), observer ratings (e.g., staff ratings, a feeling checklist), and oral (e.g., a critical-incidents audiotape)—were used to assess knowledge acquisition, counseling effectiveness, and generalization to personality and behavior functioning. In comparison with the attention group, the skills group showed significantly greater written knowledge acquisition on the CCI and significantly greater use of counseling skills in oral responses to critical-incidents audiotapes and in counseling interviews. Effects did not generalize to a personality measure or to ward behavior, nor did increased use of counseling skills affect helpee self-disclosure in peer interviews. Longer training periods may be needed to effect broader changes. (26 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of expert and referent influence, physical attractiveness, and gender on perceptions of counselor attributes.

Investigated the predictions of social influence theory with respect to Ss' perceptions of counselors' professional and personal attributes and Ss' evaluations of the counselors. Expert and referent power bases and influence attempts were crossed with levels of counselor physical attractiveness together with S and counselor gender in a counseling analog. 64 male and 64 female undergraduates viewed 1 of 8 videotaped simulated counseling interviews and completed rating forms assessing their perceptions and evaluations. Analyses supported social influence theory in that (a) expert conditions produced higher ratings of professional attributes than did referent conditions, (b) referent conditions produced higher ratings of personal attributes than did expert conditions, (c) congruent conditions (power base and influence attempt) produced higher ratings than did incongruent conditions on all measures, and (d) physical attractiveness mediated the effects of congruence for personal but not for professional attributes. No gender differences were present. Implications for the social influence process are discussed. (26 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparing the effectiveness of two directive styles in the academic counseling of foreign students.

Asian foreign students (34 male and 16 female 19–44 yr olds) participated in a comparative study on the effectiveness of 2 directive academic counseling styles (authoritative vs collaborative) on foreign students' ratings of peer counselor effectivenss. A within-group variable, acculturation level, was also examined. High-acculturated foreign students rated authoritative peer counselors higher in overall effectiveness, whereas low-acculturated foreign students rated collaborative peer counselors higher. Results support the importance of taking into account within-group differences among culturally diverse clients when conducting research. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Age-related differences in children's perceptions of social deviance: Changes in behavior or in perspective?

Previous research by the present authors (see record 1985-14217-001) revealed grade-related changes in children's ratings of aggression and withdrawal in peers. The contributions to such changes of age-related differences in the perspective of the raters and in the behavior of the children rated were investigated. Study 1 examined 120 teachers' ratings of aggression and withdrawal in 1st-, 4th-, and 7th-grade children to assess effects of age of children rated. In contrast to earlier findings with peer raters, no differences were found across grade level in the organization of teacher ratings. Study 2 examined age of rater differences in 436 1st-, 4th-, and 7th-grade Ss' beliefs about behavior that might be displayed by hypothetical peers. Differences paralleled those observed earlier in children's actual peer ratings. Study 3 examined 351 1st- and 7th-grade Ss' ratings of peers who were older or younger than the raters to assess the influence of age of rater on Ss' ratings. Age of rater effects emerged even when Ss rated peers who were not their age mates. These findings suggest that differences across grade level reported in children's peer ratings largely reflect differences in the child raters' view of behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of anger management training with mentally retarded adults in group treatment.

Conducted a components analysis of a cognitive-behavioral anger management program with 54 mildly or moderately mentally retarded 17–57 yr olds attending vocational training programs. Self-control training was given in 1 of 4 groups: relaxation training, self-instruction, problem solving, or a combined anger management condition. Dependent measures included self-reports, ratings of videotaped role-plays, and supervisor ratings. Results reveal decreases in aggressive responding over time and no significant between-group differences. The study suggests that anger management training with mentally retarded adults may be effective. (8 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A cost/benefit analysis of the interpersonal process recall technique.

Examined interpersonal process recall (IPR), a supervisory technique, by a cost-benefit analysis. Cost was defined as the possible inhibitory effects on client self-exploration of the use of IPR; benefit was defined as client satisfaction, increased supervisor ratings, and increased counselor empathy levels. Counselors were 36 master's level trainees, clients were 36 undergraduate psychology students, and supervisors were 6 doctoral students in counseling. The IPR treatment consisted of 3 videotaped interviews followed by client, counselor, or mutual recall. The comparison treatment consisted of 3 audiotaped interviews followed by traditional supervision. Results indicate that IPR, when compared to traditional supervision, did not produce differential effects on counselors' empathy level, client satisfaction, supervisor ratings, or clients' self-reported inhibition. Method of supervision, however, did effect significant change in clients' level of self-exploration over time. (32 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Team effectiveness: Beyond skills and cognitive ability.

On the basis of job analysis results, the validity of using measures of general cognitive ability, job-specific skills, and personality traits jointly at both the individual level and the group level to predict the performance of 79 four-person, human resource work teams was evaluated. Team member trait and job skill scores were aggregated with a conjunctive model of task performance. At the individual level of analysis, measures of personality (i.e., Agreeableness and Conscientiousness) predicted peer ratings of team member performance beyond measures of job-specific skills and general cognitive ability. Similarly, at the group level of analysis, both Agreeableness and Conscientiousness predicted supervisor ratings of work team performance, objective measures of work team accuracy, and work completed. At both the individual and group levels, the trait of Agreeableness predicted Interpersonal Skills. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A new look at within-source interrater reliability of 360-degree feedback ratings.

This study investigated within-source interrater reliability of supervisor, peer, and subordinate feedback ratings made for managerial development. Raters provided 360-degree feedback ratings on a sample of 153 managers. Using generalizability theory, results indicated that little within-source agreement exists; a large portion of the error variance is attributable to the combined rater main effect and Rater X Ratee effect; more raters are needed than currently used to reach acceptable levels of reliability; supervisors are the most reliable with trivial differences between peers and subordinates when the numbers of raters and items are held constant; and peers are the most reliable, followed by subordinates, followed by supervisors, under conditions commonly encountered in practice. Implications for the validity, design, and maintenance of 360-degree feedback systems are discussed along with directions for future research in this area. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Session impact, working alliance, and treatment outcome in brief counseling.

Examined associations among working alliance, session evaluations over the course of brief counseling, and counseling outcome. At a training clinic, 61 clients and student counselors from 41 dyads completed ratings of session depth, smoothness, positivity, and arousal after their 1st 12 sessions. Alliance ratings were completed after the 4th session and at termination. Clients and counselors also provided 5 measures of counseling outcome. Results indicated that session evaluations were significantly related to later alliance ratings, and positive alliance ratings predicted subsequent session evaluations. Both linear and curvilinear session effects were suggested. Associations for client ratings were generally stronger than for counselor ratings. Both alliance and session evaluations were significant unique predictors of client-related outcome, but only alliance predicted counselor-rated outcome. (PsycINFO Database Record (c) 2010 APA, all rights reserved)