rwg: An assessment of within-group interrater agreement.

F. L. Schmidt and J. E. Hunter (1989) critiqued the within-group interrater reliability statistic (rwg) described by L. R. James et al (1984). S. W. Kozlowski and K. Hattrup (1992) responded to the Schmidt and Hunter critique and argued that rwg is a suitable index of interrater agreement. This article focuses on the interpretation of rwg as a measure of agreement among judges' ratings of a single target. A new derivation of rwg is given that underscores this interpretation. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit.

A previously described coefficient of agreement for nominal scales, kappa, treats all disagreements equally. A generalization to weighted kappa (Kw) is presented. The Kw provides for the incorpation of ratio-scaled degrees of disagreement (or agreement) to each of the cells of the k * k table of joint nominal scale assignments such that disagreements of varying gravity (or agreements of varying degree) are weighted accordingly. Although providing for partial credit, Kw is fully chance corrected. Its sampling characteristics and procedures for hypothesis testing and setting confidence limits are given. Under certain conditions, Kw equals product-moment r. The use of unequal weights for symmetrical cells makes Kw suitable as a measure of validity. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Evaluation of the structures of agreement and disagreement in reliability studies

AIMS: To investigate, in healthy volunteers, the relationships between the plasma concentrations (C, ng ml(-1)) of zabiciprilat, the active metabolite of the angiotensin I-converting enzyme inhibitor (ACEI) zabicipril, and the effects (E) induced on plasma converting enzyme activity (PCEA, nmol ml(-1) min(-1)), brachial and femoral artery flows (BAF, FAF, ml min(-1)), and brachial and femoral vascular resistances (BVR, FVR, mmHg x s ml(-1)) after a single oral administration of two doses (0.5 and 2.5 mg) of zabicipril. METHODS: The study was placebo-controlled, randomized, double-blind and crossover. E was related to C by the Hill model, E = Emax x Cgamma/(CE50gamma + Cgamma), fitted to the data of both doses simultaneously. RESULTS: We obtained (mean +/- s.d.) Emax = -99 +/- 1%, CE50 = 2.2 +/- 1.0 ng ml(-1) and gamma = 1.0 +/- 0.4 for PCEA, Emax = 55 +/- 26 ml min(-1), CE50 = 5.1 +/- 4.0 ng ml(-1) and gamma = 2.4 +/- 1.6 for BAF, and Emax = -45 +/- 10%, CE50 = 2.0 +/- 1.3 ng ml(-1) and gamma = 2.3 +/- 1.4 for BVR. The parameters obtained for FAF and FVR were similar to those obtained for BAF and BVR, respectively. The CE95 (C required to induce 95% of Emax) varies from 7 to 17 ng ml(-1) for haemodynamic effects. CONCLUSIONS: As zabiciprilat peak plasma concentrations average 20 ng ml(-1) after the 2.5 mg dose of zabicipril, this dose of the drug should be sufficient to induce optimal haemodynamic effects.     

Premature consensus on consensus? Effects of sample-based versus self-based consensus information.

Evaluated the combined effect of self-based (knowledge of one's own behavior) and sample-based information (knowledge of others' behavior) and investigated R. D. Hansen and J. M. Donoghue's (see record 1979-23605-001) conclusion that population estimates and trait attributions are more affected by self-based than sample-based consensus. 84 paid Ss assigned to a self- or sample-based group responded to a pretested comedy tape or to the tape and 2 target people's opinions (funny/not funny) of the tape. Results show that either self- or sample-based consensus alone affected population estimates and trait attributions. For Ss with self-based consensus, additional congruent sample information enhanced and conflicting sample information attenuated the effects produced by self-based consensus. Thus Hansen and Donoghue's conclusion that one's own behavior is taken as the norm despite the contrary behavior of others is demonstrably premature. (18 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Personality coherence: Moderating self–other profile agreement and profile consensus.

Traditional research on moderator variables in personality has focused on measures of relative consistency. In contrast, using Goldberg's (1992) adjectives representing the Big Five personality traits, the authors examined the applicability of moderator variables to measures of personality coherence. The authors considered 3 traditional moderator variables (interitem variability, construct similarity, and scalability) and one new moderator variable: the temporal stability of response patterns. Across 2 studies, individuals with temporally stable response patterns had higher levels of personality coherence, as measured by self-other profile agreement and informant profile consensus, than did individuals with less temporally stable patterns. By comparison, the normatively based moderator variables did not moderate self-other profile agreement and informant profile consensus. The implications for personality structure and coherence are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Intergroup consensus/disagreement in support of group-based hierarchy: An examination of socio-structural and psycho-cultural factors.

A meta-analysis examined the extent to which socio-structural and psycho-cultural characteristics of societies correspond with how much gender and ethnic/racial groups differ on their support of group-based hierarchy. Robustly, women opposed group-based hierarchy more than men did, and members of lower power ethnic/racial groups opposed group-based hierarchy more than members of higher power ethnic/racial groups did. As predicted by social dominance theory, gender differences were larger, more stable, and less variable from sample to sample than differences between ethnic/racial groups. Subordinate gender and ethnic/racial group members disagreed more with dominants in their views of group-based hierarchy in societies that can be considered more liberal and modern (e.g., emphasizing individualism and change from traditions), as well as in societies that enjoyed greater gender equality. The relations between gender and ethnic/racial groups are discussed, and implications are developed for social dominance theory, social role theory, biosocial theory, social identity theory, system justification theory, realistic group conflict theory, and relative deprivation theory. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

A realistic test of individual versus group consensus decision making.

Nearly all research on the accuracy of individual versus group decision making has used ad hoc groups, artificial problems, and trivial or nonexistent reward contingencies. These studies have generally concluded that the knowledge base of the most competent group member appears to be the practical upper limit of group performance and that process gains will rarely be achieved. We studied individual versus group decision making by using data from 222 project teams, ranging in size from 3 to 8 members. These teams were engaged in solving contextually relevant and consequential problems and, in direct contrast with previous research, the groups outperformed their most proficient group member 97% of the time. Furthermore, 40% of the process gains could not be explained by either average or most knowledgeable group member scores. Implications for management practice are also discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Item content versus item wording: Disentangling role conflict and role ambiguity.

The Role Ambiguity Scale and Role Conflict Scales (J. R. Rizzo et al [1970]) assess ambiguity with 6 negatively worded items and conflict with 8 positively worded items, respectively. This methodological confound between item wording and content precludes unambiguous interpretation. In the present study, confirmatory factor analysis of these 2 scales and the Role Overload Scale (T. A. Beehr et al; see record 1976-11156-001) which has positively and negatively worded items was used to disentangle this confound. Across 2 independent samples (N?=?767 and N?=?363), a 3-factor model consistent with conceptual definitions of role ambiguity, conflict, and overload fit the data better than models with (a) one general role-stress factor, (b) a general role-stress and a method (item wording) factor, or (c) two method (positive and negative wording) factors. These results support the construct validity of Rizzo's scales; the consistency of the results across 2 independent samples suggests their generalizability. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Self- and interpersonal evaluations: Esteem theories versus consistency theories.

Contrasts and derives different predictions from 2 social psychological theories on the relation between self- and interpersonal evaluation. Self-esteem theory predicts that the higher an individual's self-evaluation, the less his tendency to reciprocate evaluations from others, whereas self-consistency theory predicts that the higher the individual's self-evaluation the greater his tendency to reciprocate evaluations from others. Studies bearing on these predictions are reviewed, and the evidence tends to support self-esteem theory. 2 extensions of the assumptions of self-esteem theory are presented and discussed in terms of accounting for results which apparently support self-consistency theory. Esteem-theory results appear to be distinguished from consistency-theory results by whether or not the experimental S is the direct target of another person's actions. The implications of this distinction for other problems of person perception and social evaluation are discussed. It is concluded that cognitive consistency theories may be somewhat overworked as explanatory frameworks for the study of social evaluations. (59 ref.) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Extension of the Kelley attribution model: The role of comparison-object consensus, target-object consensus, distinctiveness, and consistency.

Conducted 2 experiments with 544 university students to demonstrate the desirability of refining H. H. Kelley's (see record 1973-24800-001) causal attribution model. The 1st experiment demonstrated the importance of comparison-object consensus as well as the usual factors of target-object consensus, distinctiveness, and consistency. These 4 factors all follow from a "diamond" model that delineates 4 elements (person, target object, other people, comparison object) and the possible relations between all possible pairs of these elements. The 2nd experiment described the types of causal explanations that Ss offer and studied the situations in which causal attributions are formulated. A classification scheme was developed to describe the various types of causal attributions. Ss made more causal attributions in those situations in which person causes were more salient. (36 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The self-esteem motive in social influence: agreement with valued majorities and disagreement with derogated minorities

We conducted three experiments to examine variables that might influence the longevity of socially induced food preferences in Norway rats. The duration of social influence on the food choices of 42-day-old rats (1) increased with both increasing numbers of demonstrators and increasing numbers of demonstrations by a single demonstrator, (2) varied with the temporal distribution of demonstrations, but (3) did not vary with the age of demonstrators. The results suggest that a single episode of social learning produces short-term, but not long-term, effects on a Norway rat's food choices. Copyright 1998 The Association for the Study of Animal Behaviour. Copyright 1998 The Association for the Study of Animal Behaviour.     

A personality dimension of consistency versus variability without the use of self-reports or ratings.

34 university students enacted scenes in which they portrayed junior high school mathematics teachers giving a short "lesson." Ss portrayed the teachers in 3 different ways: natural, extraverted, and introverted. Objective measures of time talking, eye contact time, and total lesson time were employed to construct cross-trial variability scores. The relationships between these cross-trial variability scores were highly significant, suggesting that an individual difference measure of intraindividual behavioral variability may be constructed without resort to ratings. Behavioral variability was also related to variability in self-ratings. (11 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Do partners agree about abuse in their relationship?: A psychometric evaluation of interpartner agreement.

This study tested whether partners can be relied on to provide congruent reports about abuse in their relationship. The authors examined whether interpartner agreement (IA) varies as a function of whether the perpetrator is the man or the woman, and by whether the abusive behavior being reported is physical or psychological. Guided by psychometric test theory, the authors examined whether weak IA about specific behaviors can be improved by aggregating behavior items into scales and by controlling for random measurement error. A representative sample of 360 young couples was studied. IA did not vary with the perpetrator's gender or with the nature of the abusive behaviors, but victims (both men and women) reported somewhat more abuse than did their perpetrators. IA about specific abusive behaviors was only poor to fair, but it became very good when items were aggregated into scales and even better when measurement errors were removed form the reports. The findings suggest that reports of abuse can be aggregated to form internally consistent scales that show strong IA, thereby fulfilling criteria for reliability. Moreover, under research conditions that guarantee confidentiality, either abuser reports or victim reports are suitable methods for use in research on partner abuse. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Thinking about parenting: Reasoning about developmental issues across the life span.

Examined reasoning about developmental problems over the life span and related it to cognitive style and parenting behaviors. In Study 1, 35 children's and adults' concepts of development (COD) stages (A. Sameroff and L. Feil, 1985) were investigated. As hypothesized, stage scores were positively related to age and to 2 other developmental indexes. In Study 2, 24 parents and their 5th-grade children were observed. COD stage scores were positively related to more effective tutoring styles by parents. In Study 3, 60 adults in 3 different age groups were interviewed about standard and personal parenting dilemmas. COD stage scores were lower for the oldest adults on the hypothetical dilemma only and were inversely correlated with a measure of right-wing authoritarianism. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

The consistency model: A reply to Parducci.

It is admitted that more research is needed to determine the variables responsible for the large individual differences in the amount of adjustment of scale to a change in stimulus frequencies. Special attention should be given to the different response selection strategies Ss use when a stimulus falls outside the remembered range of the stimuli previously judged. In the majority of cases, however, adjustment to a contextual change is rather incomplete. This indicates that there is a tendency for the initial scale to persist throughout many trials. The consistency model provides a single framework for contextual and transfer effects in absolute judgments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

What do you learn about someone over time? The relationship between length of acquaintance and consensus and self-other agreement in judgments of personality.

Theory and research examining length of acquaintance and consensus among personality judgments have predominantly examined each dimension of personality separately. In L. J. Cronbach's (1955) terminology, this trait-centered approach combines consensus on elevation, differential elevation, and differential accuracy in personality judgments. The current article extends D. A. Kenny's (1991, 1994) weighted average model (WAM)--a theoretical model of the factors that influence agreement among personality judgments--to separate out two of Cronbach's components of consensus: stereotype accuracy and differential accuracy. Consistent with the predictions based on the WAM, as length of acquaintance increased, self-other agreement and consensus differential accuracy increased, stereotype accuracy decreased, and trait-level or raw profile correlations generally remained unchanged. Discussion focuses on the conditions under which a relationship between length of acquaintance and consensus and self-other agreement among personality evaluations emerges and how impressions change over time. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Effects of communication, information overlap, and behavioral consistency on consensus in social perception.

Three experiments (N?=?69, 162, and 201, respectively) were conducted to test the mathematically derived predictions of the Weighted Average Model (D. A. Kenny, 1991) of consensus in interpersonal perception. Study 1 estimated the effect of perceiver communication, Study 2 estimated the effects of communication and stimulus overlap, and Study 3 estimated the effects of communication, overlap, and target consistency on consensus. The strongest consensus was found when perceivers communicated about highly overlapping information about targets who were cross-situationally consistent. Conversely, the lowest level of consensus was observed when perceivers did not communicate and had nonoverlapping information about targets who were cross-situationally inconsistent. Both stimulus variables (overlap and consistency) and an interpersonal variable (communication) affected consensus as predicted by the Weighted Average Model. (PsycINFO Database Record (c) 2010 APA, all rights reserved)