A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings.

Two recent reviews have attempted to summarize findings quantitatively regarding assessment center (AC) construct-related validity (i.e., Lance, Lambert, Gewin, Lievens, & Conway, 2004; Lievens & Conway, 2001). Unlike these previous studies, which reanalyzed individual multitrait-multimethod (MTMM) matrices from previously published research, the authors recoded and combined past matrices into a single MTMM matrix. This matrix, comprised of 6 dimensions each measured by 6 exercises, was then analyzed, providing a more generalizable set of results. Both dimensions and exercises were found to contribute substantially to AC ratings. Specific dimensions (i.e., communication, influencing others, organizing and planning, and problem solving) appeared more construct valid than others (i.e., consideration/awareness of others and drive). Implications for AC design and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The impact of item wording and behavioral specificity on the accuracy of direct behavior ratings (DBRs).

Direct behavior ratings (DBRs) combine aspects of both systematic direct observation and behavior rating scales to create a feasible method for social behavior assessment within a problem-solving model. The purpose of the current study was to examine whether accuracy of DBRs was affected depending on the behaviors selected to be rated using a DBR. Specifically, the impact target behavior wording (positive vs. negative) and degree of specificity by which the behaviors were defined were investigated. Participants (N = 145), assigned to 1 of 4 experimental conditions, were asked to view video clips of a classroom setting and rate target student behavior. Results indicated that the wording and specificity of behavior included on a DBR may influence the accuracy of ratings. The most accurate ratings of academically engaged behavior occurred when the behavior was defined with a global definition and positive wording. In contrast, the most accurate ratings of disruptive behavior were obtained with a global definition and either positive or negative wording. Limitations, implications, and future directions are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Validity of observer ratings of the five-factor model of personality traits: A meta-analysis.

Conclusions reached in previous research about the magnitude and nature of personality–performance linkages have been based almost exclusively on self-report measures of personality. The purpose of this study is to address this void in the literature by conducting a meta-analysis of the relationship between observer ratings of the five-factor model (FFM) personality traits and overall job performance. Our results show that the operational validities of FFM traits based on observer ratings are higher than those based on self-report ratings. In addition, the results show that when based on observer ratings, all FFM traits are significant predictors of overall performance. Further, observer ratings of FFM traits show meaningful incremental validity over self-reports of corresponding FFM traits in predicting overall performance, but the reverse is not true. We conclude that the validity of FFM traits in predicting overall performance is higher than previously believed, and our results underscore the importance of disentangling the validity of personality traits from the method of measurement of the traits. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

The developmental relations between conceptual and procedural knowledge: A multimethod approach.

Interactions between conceptual and procedural knowledge influence the development of mathematical competencies. However, after decades of research, these interrelations are still under debate, and empirical results are inconclusive. The authors point out a source of these problems. Different kinds of knowledge and competencies only show up intertwined in behavior, making it hard to measure them validly and independently of each other. A multimethod approach was used to investigate the extent of these problems. A total of 289 fifth and sixth graders’ conceptual and procedural knowledge about decimal fractions was measured by 4 common hypothetical measures of each kind of knowledge. Study 1 tested whether treatments affected the 2 groups of measures in consistent ways. Study 2 assessed, across 3 measurement points, whether conceptual and procedural knowledge could be modeled as latent factors underlying the measures. The results reveal substantial problems with the validities of the measures, which might have been present but gone undetected in previous studies. A solution to these problems is essential for theoretical and practical progress in the field. The potential of the multimethod approach for this enterprise is discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach.

A review of research on job performance suggests 3 broad components: task, citizenship, and counterproductive performance. This study examined the relative importance of each component to ratings of overall performance by using an experimental policy-capturing design. Managers in 5 jobs read hypothetical profiles describing employees' task, citizenship, and counterproductive performance and provided global ratings of performance. Within-subjects regression analyses indicated that the weights given to the 3 performance components varied across raters. Hierarchical cluster analyses indicated that raters' policies could be grouped into 3 homogeneous clusters: (a) task performance weighted highest, (b) counterproductive performance weighted highest, and (c) equal and large weights given to task and counterproductive performance. Hierarchical linear modeling indicated that demographic variables were not related to raters' weights. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Predicting future blood donor returns: Past behavior, intentions, and observer effects.

A prospective design was used to explore the efficacy of 6 factors (e.g., intentions) to predict the number of future blood donations in an initial sample of 630 blood donors. Differential predictions are made for the roles of past behavior and intentions with respect to occasional (4 or fewer previous blood donations) and regular (5 or more previous blood donations) blood donors. Intentions were predictive for occasional donors, and past behavior was predictive for regular donors. Furthermore, for regular donors only, an inverted U-shaped curve explained the relationship between past behavior and future behavior. Finally, it is reported that observing others fainting produces a reduction in the number of future donations for occasional donors. The theoretical and practical implications of these findings are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The utility of the reasonable woman legal standard in hostile environment sexual harassment cases: A multimethod, multistudy examination.

The case of Ellison v. Brady (1991) was a landmark decision in establishing a new legal standard-the reasonable woman standard-for evaluating hostile environment allegations of sexual harassment and in relying, albeit indirectly, on social science evidence to support its decision. The authors review the legal standard in sexual harassment law, and the legal arguments and social science evidence underlying Ellison. The new standard could affect judgments broadly by sensitizing jurors to the situation of a female plaintiff, or it could affect judgments selectively by sensitizing only those people who might otherwise be especially insensitive to her situation. Five studies that focus on the effect of the reasonable woman standard on people's assessments of hostile environment sexual harassment indicate that the reasonable woman standard has modest, if any, effects on the judgments studied. Implications are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Relationships between job difficulty, employee's attitude toward his job, and supervisory ratings of the employee effectiveness.

Using correlational techniques, an investigation was made of the relationships between job difficulty (estimated by job evaluation factors), employee attitudes toward his job, and job environment, and supervisory ratings of employee performance. As job difficulty increased, employee attitudes were significantly more positive toward the job, management, and communication, and opportunity for advancement. Partial correlations showed that the relationship between job satisfaction and job difficulty increased when the effects of general morale were eliminated. Supervisory ratings of employee effectiveness were significantly rated (negatively) to employee salary and job tenure. Correlations between employee's attitude dimensions indicate increasing complexity of job content and increased content with people as a part of the job, are positively related to an employee's attitude toward his job. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Understanding knowledge effects on attitude-behavior consistency: The role of relevance, complexity, and amount of knowledge.

The role of properties of attitude-relevant knowledge in attitude- behavior consistency was explored in 3 experiments. In Experiment 1, attitudes based on behaviorally relevant knowledge predicted behavior better than attitudes based on low-relevance knowledge, especially when people had time to deliberate. Relevance, complexity, and amount of knowledge were investigated in Experiment 2. It was found that complexity increased attitude- behavior consistency when knowledge was of low-behavioral relevance. Under high-behavioral relevance, attitudes predicted behavior well regardless of complexity. Amount of knowledge had no effect on attitude- behavior consistency. In Experiment 3, the findings of Experiment 2 were replicated, and the complexity effect was extended to behaviors of ambiguous relevance. Together, these experiments support an attitude inference perspective, which holds that under high deliberation conditions, people consider the behavioral relevance and dimensional complexity of knowledge underlying their attitudes before deciding to act on them. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Division of Health Psychology 1986 Student Research Award: Type A behavior and cognitions: Is hostility the bad actor?

The following abstract is from the award winning paper submitted for the Division of Health Psychology 1986 Student Research Award Competition. In this study, the authors found that behavior type was significantly associated with cardiac damage and hostile cognitions. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The moderating influence of job performance dimensions on convergence of supervisory and peer ratings of job performance: Unconfounding construct-level convergence and rating difficulty.

Meta-analysis was used to cumulate the correlations between supervisor and peer ratings for different dimensions of job performance. The general pattern in the data suggested that raters from the same organizational level disagree as much as raters from different levels. Methods were used to separate the effects of rating difficulty and lack of construct-level convergence on the correlation between supervisor and peer ratings. The authors found complete construct-level convergence for ratings of overall job performance, productivity, effort, job knowledge, quality, and leadership but not for ratings of administrative competence, interpersonal competence, and compliance or acceptance of authority. Higher rating difficulty was more strongly associated with lower mean observed peer-supervisor correlations than were construct-level disagreements between peers and supervisors. Implications for research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The classroom environment and students' reports of avoidance strategies in mathematics: A multimethod study.

The relation between the learning environment (e.g., students' perceptions of the classroom goal structure and teachers' instructional discourse) and students' reported use of avoidance strategies (self-handicapping, avoidance of help seeking) and preference to avoid novelty in mathematics was examined. Quantitative analyses indicated that students' reports of avoidance behaviors varied significantly among classrooms. A perceived emphasis on mastery goals in the classroom was positively related to lower reports of avoidance. Qualitative analyses revealed that teachers in high-mastery/low-avoidance and low-mastery/high-avoidance classrooms used distinctively different patterns of instructional and motivational discourse. High incidence of motivational support was uniquely characteristic of high-mastery/ low-avoidance classrooms, suggesting that mastery goals may include an affective component. Implications of the results for both theory and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Fathers' ratings in the assessment of their child's anxiety symptoms: A comparison to mothers' ratings and their associations with paternal symptomatology.

We examined the comparability between mothers' and fathers' ratings in the assessment of their child's anxiety symptoms. The sample consisted of 78 youth (6 to 17 years) and their mothers and fathers who presented to a childhood anxiety disorders specialty research clinic. Using intraclass correlation coefficients, mother?father agreement of their child's anxiety symptoms was found to be moderate. Mean differences between mothers' and fathers' ratings of their child's anxiety were not significantly different. Both maternal and paternal self-ratings of psychopathology predicted respective ratings of their child's anxiety. Although either mothers or fathers can provide useful information, use of multiple informants is encouraged, especially when parental psychopathology is present. Additional implications and suggestions for future research are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effects of school-based intervention programs on aggressive behavior: A meta-analysis.

Research on the effectiveness of school-based programs for preventing or reducing aggressive behavior was synthesized with a meta-analysis. Changes in aggressive behavior between pretest and posttest were analyzed for developmental patterns and characteristics associated with differential effects. Control groups showed little change in aggressive behavior, but there were significant reductions among intervention groups. Most studies were conducted on demonstration programs; the few studies of routine practice programs showed much smaller effects. Among demonstration programs, positive outcomes were associated with a variety of study, subject, and intervention characteristics. Most notably, higher risk youth showed greater reductions in aggressive behavior, poorly implemented programs produced smaller effects, and different types of programs were generally similar in their effectiveness, other things equal. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The context of aggressive behavior in marriage: A longitudinal study of newlyweds.

Under what circumstances are spouses more or less likely to engage in aggressive behaviors? To address this question, the current study drew on multiple longitudinal assessments of 1st-married newlyweds to examine correlates of within-subject variability in aggressive behavior. Controlling for marital satisfaction, the authors found that spouses were more likely to engage in physical aggression at times when they engaged in higher levels of psychological aggression. Additionally, husbands reporting higher levels of chronic stress were more likely to engage in physical aggression overall and were more likely to engage in physical aggression when they were experiencing higher than average levels of acute stress. These results highlight how demands and supports in the context external to a marriage may affect processes within the marriage. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Parenting and antisocial behavior: A model of the relationship between adolescent self-disclosure, parental closeness, parental control, and adolescent antisocial behavior.

This study used data collected from a sample of 840 Italian adolescents (418 boys; M age = 12.58) and their parents (657 mothers; M age = 43.78) to explore the relations between parenting, adolescent self-disclosure, and antisocial behavior. In the hypothesized model, parenting practices (e.g., parental monitoring and control) have direct effects on parental knowledge and antisocial behavior. Parenting style (e.g., parent–child closeness), on the other hand, is directly related to adolescent self-disclosure, which in turn is positively related to parental knowledge and negatively related to adolescents’ antisocial behavior. A structural equation model, which incorporated data from parents and adolescents, largely supported the hypothesized model. Gender-specific models also found some gender differences among adolescents and parents, as the hypothesized model adequately fit the subsample of mothers but not fathers. Mothers’ closeness to girls predicted their knowledge of their daughters’ behavior; mothers’ control predicted boys’ antisocial behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The Impact of Job Complexity and Performance Measurement on the Temporal Consistency, Stability, and Test-Retest Reliability of Employee Job Performance Ratings.

Although research has shown that individual job performance changes over time, the extent of such changes is unknown. In this article, the authors define and distinguish between the concepts of temporal consistency, stability, and test-retest reliability when considering individual job performance ratings over time. Furthermore, the authors examine measurement type (i.e., subjective and objective measures) and job complexity in relation to temporal consistency, stability, and test-retest reliability. On the basis of meta-analytic results, the authors found that the test-retest reliability of these ratings ranged from .83 for subjective measures in low-complexity jobs to .50 for objective measures in high-complexity jobs. The stability of these ratings over a 1-year time lag ranged from .85 to .67. The analyses also reveal that correlations between performance measures decreased as the time interval between performance measurements increased, but the estimates approached values greater than zero. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Do People Agree About the Causes of Behavior? A Social Relations Analysis of Behavior Ratings and Causal Attributions.

Two studies examined consistency and agreement in behavior ratings and causal attributions. In Study 1, participants (N = 280) engaged in a series of getting-acquainted conversations in one of 3 communication media (face-to-face, telephone, computer mediated); in Study 2, participants (N = 120) engaged in a competitive group task. In both studies, participants rated themselves and their interaction partners on a set of behaviors and then made attributions about the causes of those behaviors. The major findings were that (a) participants consistently favored some causal factors over others in explaining both their own and their partners' behavior, supporting the existence of generalized attributional styles; and (b) participants showed moderate self-partner and partner-partner agreement about behavior but virtually no agreement about the causes of behavior. Thus, in brief interactions people tend to see themselves and others through the lens of their stable patterns of perceiving and interpreting behavior. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The effect of subject liberalism, anonymity, and race of experimenter on subjects' ratings of oriental and white photographs.

Liberal and conservative subjects assigned adjective trait ratings to photographs of oriental and white persons. Ratings were made with either an oriental or white experimenter present and in conditions where ratings were made either anonymously or not. It was found that both liberal and conservative subjects rated oriental photographs more positively than white photographs. This tendency was more pronounced for liberal subjects. When ratings were made anonymously the oriental-white rating discrepancy diminished for liberal subjects and was reversed for conservative subjects. Liberals who responded anonymously increased their oriental-white rating discrepancy when the experimenter was oriental. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Implicational schemata and the correspondence bias: On the diagnostic value of situationally constrained behavior.

Drawing on G. D. Reeder's (1993) schematic model of dispositional inference, it is hypothesized that perceivers' tendency to draw correspondent dispositional inferences from situationally constrained behavior (i.e., the correspondence bias) can be due to the application of schematic assumptions about trait-behavior relations (i.e., implicational schemata) within the process of situational adjustment. Applied to attitude attribution, situational adjustment is hypothesized to follow an implicit theory of ability, implying that only authors with a corresponding attitude are able to write a persuasive essay toward a given position. Results from 6 experiments offer converging evidence for this hypothesis. Implications for a sufficient understanding of the processes that lead to the correspondence bias are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)