Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit

OBJECTIVE: Compare resident evaluations by self, nurses, and attending physicians. DESIGN: Prospective cohort. SETTING: University intensive care unit. SUBJECTS: Sixty residents. INTERVENTIONS: End-rotational evaluation using a standardized, multiple-choice examination and one of two subjective instruments, Global Rating Scale and Behaviorally Anchored Rating Scale. MEASUREMENTS AND MAIN RESULTS: Means for overall competence, using both the Behaviorally Anchored Rating Scale and the Global Rating Scale clustered between 3 to 4 on a 5-point scale. Physicians' evaluations correlated with the multiple-choice test scores (Spearman's rho 0.3082, p = .005, n = 82), whereas neither self-evaluation (Spearman's rho 0.1124, p = .65, n = 42) nor nurses' evaluations (Spearman's rho 0.2060, p = .069, n = 79) had a significant correlation with test scores. Spearman's correlations were not significant for either overall competence or specific medical knowledge by any category of evaluator using the Global Rating Scale. Spearman's rho correlations and kappa statistic between the three types of evaluators (physicians, nurses, and self) for each criterion of the Behaviorally Anchored Rating Scale demonstrated significant correlations between the ratings of physicians and nurses, except for the assessment of humanistic qualities. Pooled clinical skills-history taking (b = 0.277, p <.009), humanistic qualities (b = 0.607, p <.000), and professional attitudes and behavior (b = 0.488, p < .000) systematically differed in ratings comparing self with nurse and physician (by analysis of variance). The explanatory power of the model of ratings (independent variables of year of residency, category of evaluator, evaluation criteria, and interaction terms) was 47.3% (r2adj). CONCLUSIONS: Self-rating by residents did not correlate to multiple-choice test scores and differed in some criteria with physicians' or nurses' evaluations. We found many similarities and some differences between physicians' and nurses' evaluations of residents. We speculate that different categories of evaluators assess different aspects of performance. Assessment by a varied group of evaluators should be used when attempts to predict future practice are made.