Effects of rater training and diary-keeping on psychometric error in ratings.

Compared the psychometric properties of ratings on behavioral expectation scales (BES) across 4 groups totalling 156 undergraduate raters. Groups differed with respect to amount of prior training (1 hr or more), the nature of psychometric errors, and the extent of exposure to scales (read scales and recorded observed critical incidents, discussed general scale dimensions, or no exposure to scales). Three Ss from each group rated 1 of 13 instructors during the last week of a 10-wk term. Significantly less leniency error and halo effect, plus higher interrater reliability, were found for the group that had received the hour of training and full exposure to the BES. Ss who had received only training had significantly less halo error than those that had received no training. The need for rater training prior to observation and the use of BES as a context for observation are discussed. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)