Do behavioral observation scales measure observation?

G. Latham and K. Wexley (see record 1980-02200-001) have claimed that behavioral observation scales (BOS) pose a simpler task for the rater than do either behaviorally anchored rating scales or graphic rating scales; with BOS, the rater need only observe and record behavior and need not make complex judgments about performance. Research on person memory suggests that recall for behaviors is structured by the same trait inferences and judgments that BOS are designed to avoid. In 2 experiments, 91 undergraduates rated videotaped lectures; data from the 1st experiment were used to construct BOS measuring clarity and speaking style. In the 2nd experiment, Ss used the BOS and a graphic rating scale to rate videotaped lectures in immediate and delayed rating conditions. As expected, the correlations between BOS ratings and judgmental ratings of performance were stronger when demands were placed on rater's recall. It is suggested that recall of behaviors is determined by the degree to which certain behaviors are representative of general judgments made about Ss being rated, and that BOS measure traitlike judgments rather than behavioral observation. (27 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Comparative analysis of goal-setting strategies across cultures.

Only a few studies that have examined the effects of participation on an individual's goal acceptance and performance have been conducted within a cross-cultural context. In the present study, we tested for the contingency between the effectiveness of goal-setting strategies and cultural values. We examined three goal-setting strategies within three different cultural groups—assigned goals, goals participatively set by a group representative and the experimenter, and goals participatively set by a group. The three cultural groups studied were U.S. students (n?=?60), individualistic and having a high power distance; Israeli students from urban areas (n?=?60), collectivistic and having a low power distance; and Israeli students from kibbutzim (n?=?60), highly collectivistic and having a low power distance. Results indicated that participative strategies led to higher levels of goal acceptance and performance than the assigned strategy. Culture did not moderate the effect of goal-setting strategies on goal acceptance, but it appeared to moderate the strategy on performance for extremely difficult goals. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effect of appraisal instrument on managerial perceptions of fairness and satisfaction with appraisals from their peers.

Examined whether the type of appraisal instrument (behavioral observation scales [BOS], behaviorally anchored rating scales [BARS], trait scales, or using no formal appraisal instrument) affected satisfaction with a peer appraisal and perceptions of fairness. 91 managers, while working in teams on a simulated task, provided one another with feedback. Satisfaction with peer appraisals was higher when BOS, BARS, or no formal instrument (control) was used to give feedback than was the case with a trait scale. Procedural justice was perceived as higher when either BOS or no instrument was used to give feedback than when the feedback was based on a trait scale. Trait scales were not perceived to be an acceptable instrument by peers for assessing their performance. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Goal setting: An extension of the research.

Investigated whether goal-setting theory generalizes to other cultures by looking at the relation between goal setting and output among 92 Caribbean women who performed home-based piecework. Results confirm the hypothesis that setting specific and difficult goals would result in higher performance than no goal or a "do one's best" goal. (5 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Behaviorally anchored scales for rating the performance of police officers.

Performance appraisal of municipal police officers has presented problems similar to those encountered in industrial-performance evaluation. The present study was concerned with the development of peer and supervisory rating scales for patrol officers. 58 police departments cooperated in constructing and field testing 8 supervisory and 9 peer rating scales. The psychometric characteristics of the scales indicated that they were suitable for use in evaluating the performance of patrol officers. Principal-components analyses indicated that 3 linear components described the variation in ratings for both supervisory and peer scales. While these components seemed to represent similar constructs, they were represented by performance dimensions specific to the level of observation. The generalizability of behaviorally anchored rating scales to settings other than those in which they were constructed is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The role of task-goal attributes in employee performance.

Many studies have demonstrated the relatively successful performance implications of formalized goal-setting programs in organizations. However, these findings typically do not identify the specific factors behind such techniques that are largely responsible for their success. Toward this end, research relating to 6 factor analytically derived attributes of employees' task goals is reviewed to ascertain which attributes are more consistently related to performance. The 6 task-goal attributes are goal specificity, participation in goal setting, feedback, peer competition, goal difficulty, and goal acceptance. Although goal specificity and goal acceptance were found to be most consistently related to performance, several intervening variables emerged that tended to affect significantly the impact of certain attributes on performance. Findings are discussed within a motivational framework. It is argued, based on the data, that performance under goal-setting conditions is a function of at least 3 important variables: the nature of the task goals, additional situational-environmental factors, and individual differences. (3 p ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Importance of participative goal setting and anticipated rewards on goal difficulty and job performance.

In Study 1, 76 engineers/scientists either participated in the setting of, or were assigned, specific behavioral goals during their performance appraisal. Participative goal setting resulted in more difficult goals being set than was the case when the goals were assigned. Perceptions of goal difficulty, however, were not significantly different in the 2 goal-setting conditions. In Study 2, the analysis of the performance data collected 6 mo later on 132 engineers/scientists revealed main effects for both goal setting and anticipated rewards. Only participative goal setting led to significantly higher performance than a "do your best" and a control group condition. There was no significant difference between the performance of the latter 2 conditions despite the fact that the individuals in the do-your-best group received knowledge of results. (20 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Bias in Behaviorally Anchored Rating Scales: Global or scale-specific?

Murphy and Constans (1987) showed that behavioral anchors can be a source of bias in ratings but did not determine whether bias in ratings of one dimension would spread to rating of other performance dimensions. In two separate studies, we manipulated behavioral anchors on one of the scales developed by Murphy and Constans (1987) and examined the effect of bias in these scales on ratings of dimensions that varied in their conceptual and evaluative similarity to the dimension that was manipulated. Neither study showed evidence that bias in ratings of one dimension spreads to ratings of other performance dimensions. These results suggest that rating bias of the sort reported by Murphy and Constans will not have a substantial effect on performance ratings in organizations. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Goal setting and task performance: 1969–1980.

Results from a review of laboratory and field studies on the effects of goal setting on performance show that in 90% of the studies, specific and challenging goals led to higher performance than easy goals, "do your best" goals, or no goals. Goals affect performance by directing attention, mobilizing effort, increasing persistence, and motivating strategy development. Goal setting is most likely to improve task performance when the goals are specific and sufficiently challenging, Ss have sufficient ability (and ability differences are controlled), feedback is provided to show progress in relation to the goal, rewards such as money are given for goal attainment, the experimenter or manager is supportive, and assigned goals are accepted by the individual. No reliable individual differences have emerged in goal-setting studies, probably because the goals were typically assigned rather than self-set. Need for achievement and self-esteem may be the most promising individual difference variables. (3? p ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Distributional ratings of performance: Further examination of a new rating format.

Examined several aspects of construct validity evidence for a distributional format by comparing it with a Behavioral Observation Scale (BOS) rating form and determining whether raters were sensitive to differences in performance variability. Raters were assigned to 1 of 2 instructional procedures, 1 of 2 form orderings (BOS or distributional rating first), and 1 of 5 conditions of performance variability. Ss rated an instructor's performance after viewing 4 videotaped excerpts of his lectures. Mean ratings were lower using the distributional format relative to the BOS format. The distributional ratings indicated that Ss were sensitive to the different variability conditions. The potential of distributional ratings for providing a richer source of performance information than more traditional ratings is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Sex effects on performance ratings on manager–subordinate dyads: A field study.

Tested the hypothesis of N. Schmitt et al (see record 1980-33528-001) that people rate those similar to themselves with more confidence, which is reflected in larger variances in performance ratings. 286 manager–subordinate dyads in all 4 sex combinations comprised the sample. Subordinates rated managers on the Behavioral Observation Scales (BOS), and managers rated subordinates using the Minnesota Satisfactoriness Scales (MSS). Bartlett's test for homogeneity of variance revealed that female subordinates produced significantly more variability when rating their male managers than when rating their female managers on total BOS rating. Female managers produced more variable ratings of male subordinates than for female subordinates on total MSS rating. Male subordinates and managers did not produce more variable BOS or MSS ratings when appraising other males than when rating females. Results are discussed in terms of role expectations. (24 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The scalability of estimated worker requirements.

Construction of rating scales and check lists in job analysis can be done with the help of people who make judgments about dimensions for the categories. Can this scalability be made acceptable, and is doing it a general ability or one that is highly specific to the requirement? "Seven experienced and trained job analysts of the United States Employment Service rated 50 jobs on 33 requirements grouped into three classes: (a) aptitudes, (b) interests, and (c) personality. Analysts were provided with definitions of each requirement, together with 'bench mark' jobs to serve as guiding examples." Most of (a), ? of (b), and about ? of (c) were scalable. "… analyst sensitivity is not a general ability but is highly specific to the requirement." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Multitrait–multimethod–multirater analysis of performance ratings for law enforcement personnel.

144 deputy sheriffs were rated on 9 job performance dimensions with 2 rating scales by 2 raters. Results indicate that the rating scales (the Multiple Item Appraisal Form and the Global Dimension Appraisal Form) developed in this study were able to minimize the major problems often associated with performance ratings (i.e., leniency error, restriction of range, and low reliability). A multitrait/multimethod analysis indicated that the rating scales possessed high convergent and discriminant validity. A multitrait/multirater analysis indicated that although the interrater agreement and the degree of rated discrimination on different traits by different raters were good, there was a substantial rater bias, or strong halo effect. This halo effect in the ratings, however, may really be a legitimate general factor rather than an error. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Task complexity as a moderator of goal effects: A meta-analysis.

Much evidence exists that supports the use of goal setting as a motivational technique for enhancing task performance; however, little attention has been given to the role of task characteristics as potential moderating conditions of goal effects. Meta-analysis procedures were used to assess the moderator effects of task complexity for goal-setting studies conducted from 1966 to 1985 (n?=?125). The reliability of the task complexity ratings was .92. Three sets of analyses were conducted: for goal-difficulty results (hard vs. easy), for goal specificity–difficulty (specific difficult goals vs. do-best or no goal), and for all studies collapsed across goal difficulty and goal specificity–difficulty. It was generally found that goal-setting effects were strongest for easy tasks (reaction time, brainstorming), d?=?.76, and weakest for more complex tasks (business game simulations, scientific and engineering work, faculty research productivity), d?=?.42. Implications for future research on goal setting and the validity of generalizing results are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A comparison of behavioral expectation scales and graphic rating scales.

Compared ratings derived from behavioral expectation scales developed by 147 personnel management students with ratings based on graphic rating scales. The ratees were 4 college professors, and the raters were the 183 students in their classes. The behaviorally anchored scales resulted in less halo error, or alternatively, more independence in ratings of different dimensions of performance. The behaviorally anchored scales did not correct for leniency in ratings. These results were observed both among raters who participated in developing the behavioral expectation scales and among similar raters who did not take part in this process. The factor structures of the 2 rating formats were essentially equivalent in "cleanness." Neither solution was judged superior to the other. However, the behavioral expectation scale format possessed greater discriminant validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relation of effort-performance expectancy to performance in goal-setting experiments.

Explored the relationship between effort–performance expectancy and task performance in goal-setting research. A review indicates that the literature in this area is highly inadequate, failing to distinguish between the expectancy–performance relation within as compared to between assigned goal conditions. Data from 2 recent experiments by the present author (see PA, Vols 68:360 and 69:11929) were reanalyzed through hierarchical multiple regression of performance on both goal level and expectancy, measured as a subjective probability of success. Ss in each experiment (86 and 58 undergraduates, respectively) performed repeated trials of a creativity task under different assigned goals. The reanalyses suggest that both goal levels and expectancy made independent contributions to variance in performance. Potential strategies for increasing performance by raising both goal levels and expectancies simultaneously are briefly discussed. New ways of conceptualizing expectancy and valence in goal-setting research are presented. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Erratum to Latham, Mitchell, and Dossett.

Reports an error in the original article by Gary P. Latham, Terence R. Mitchell, and Dennis L. Dossett (Journal of Applied Psychology, 1978, Vol. 63, No. 2, pp. 163-171). In Table 1 of the article, the mean for assigned goal setting and public recognition is incorrect. The corrected mean is given. (The following abstract of this article originally appeared in record 1979-02470-001.) In Study 1, 76 engineers/scientists either participated in the setting of, or were assigned, specific behavioral goals during their performance appraisal. Participative goal setting resulted in more difficult goals being set than was the case when the goals were assigned. Perceptions of goal difficulty, however, were not significantly different in the 2 goal-setting conditions. In Study 2, the analysis of the performance data collected 6 mo later on 132 engineers/scientists revealed main effects for both goal setting and anticipated rewards. Only participative goal setting led to significantly higher performance than a "do your best" and a control group condition. There was no significant difference between the performance of the latter 2 conditions despite the fact that the individuals in the do-your-best group received knowledge of results. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Motivation, behavior, and performance: A closer examination of goal setting and monetary incentives.

Some experimenters often test predictions from theories of motivation using performance outcomes as dependent variables. It is argued that observable behaviors that are likely to be affected by motivation manipulations should be used in combination with performance outcomes. Such procedures would be sensitive to differential effects of manipulations on various behaviors and would allow for investigation of relationships among behaviors and performance outcomes. For this experiment, 60 17–23 yr olds were hired to work individually on a 2-hr construction task. Ss were assigned to 1 of 2 pay conditions (piece-rate vs hourly) and 1 of 3 goal-setting conditions (no goal, quantity, or quality) resulting in a 2?×?3 crossed analysis of variance design. Dependent variables included 3 measures of effort, 3 measures of direction of behavior, and both quantity and quality performance. Method of payment affected quantity performance and effort. Goal setting affected quantity and quality performance and direction of behavior. Implications for designing and testing work motivation systems are discussed. (27 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evidence incompatible with a cognitive compatibility theory of rating behavior.

Assessed the cognitive complexity of 96 undergraduates with the group version of the Role Construct Repertory (REP) Test, a factor analysis of REP test data, and a sorting task. Performance ratings for 3 of the Ss' instructors were obtained with behaviorally anchored rating scales, mixed standard rating scales, graphic rating scales, and simple "alternate" 3-point rating scales. No differences in leniency, halo, or range restriction emerged either as a function of raters' cognitive complexity or a Cognitive Complexity?×?Scale Format interaction. Raters' confidence in their ratings was not associated with either cognitive complexity or rating scale format. It is concluded that researchers of performance ratings should exercise restraint before confidently conferring moderator variable status on a cognitive complexity construct. (25 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)