Interrater reliability and agreement of subjective judgments.

Indexes of interrater reliability and agreement are reviewed, and suggestions are made regarding their use in counseling psychology research. The distinction between agreement and reliability is clarified, and the relationships between these indexes and the level of measurement and type of replication are discussed. Indexes of interrater reliability appropriate for use with ordinal and interval scales are considered. The intraclass correlation as a measure of interrater reliability is discussed in terms of the treatment of between-raters variance and the appropriateness of reliability estimates based on composite or individual ratings. The advisability of optimal weighting schemes for calculating composite ratings is also considered. Measures of interrater agreement for ordinal and interval scales are described, as are measures of interrater agreement for data at the nominal level of measurement. (54 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Business is too important to be studied only by economists.

"One of the relatively major interests of the Ford Foundation at present is providing assistance in the improvement of business education. Ford has allocated more than $10,000,000.00 to this end… . For psychologists, apart from their general interest in education, the special interest comes from the emphasis put, both in the books and in the Ford Foundation's program, on the social sciences in general and on psychology in particular… . The recognition of psychology as a discipline as one of the bases from which an understanding of business as a social institution must flow makes a distinct change in the essential character of industrial psychology." (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

There is only one psychology: The dream and the reality.

Argues against J. D. Matarazzo's (see record 1988-05729-001) view that there is only one psychology. The different approaches in such fields as general, developmental, social, and clinical psychology lead to such different frames of reference that they constitute different kinds of psychologies. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Grand theories of personality cannot be integrated.

Comments on the original article "A New Big Five: Fundamental Principles for an Integrative Science of Personality," by Dan P. McAdams and Jennifer L. Pals (see record 2006-03947-002). McAdams and Pals (April 2006) presented a new model to integrate the field of personality psychology. Cultural and evolutionary factors interact with an individual's basic traits, characteristic adaptations, and life narratives, which in turn are linked to roles, demands, and behaviors. The current authors welcome McAdams and Pals's (2006) model for providing a way to integrate much of the previously disparate empirical findings in personality psychology. However, the current authors also think that McAdams and Pals (2006) overstated the inclusiveness of the model, and more generally, the current authors dispute their assertion that the grand theories of personality can be integrated within a single model. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

There is only one psychology, no specialties, but many applications.

Asserts that students who today receive a PhD degree via matriculation in a university department of psychology study the same principles, processes, and core content of the discipline regardless of the specific area of application they are pursuing. The broad content, processes, and principles that are the essence and core that differentiate psychology from physics, economics, history, and other cognate disciplines have remained the same since psychology first became a university discipline 100 years ago. It is also asserted that although many psychologists believe there currently exist bona fide specialties in psychology, neither the American Psychological Association nor most other societally relevant institutions have accorded such specialties de jure recognition. Although informal de facto recognition of specialties already exists, several other stages in the evolution of a specialty must (and very likely will) occur during the next decade before society accords specialties in psychology more formal, quasi-legal, de jure recognition. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

On various intraclass correlation reliability coefficients.

Briefly reviews the most frequently used and misused intraclass correlation--analysis of variance (ANOVA) reliability measures. Recommendations are made for the 1-way ANOVA intraclass correlation and against the use of several coefficients: B. J. Winer's (1971) so-called "anchor point method," the Kuder-Richardson Formula Number 20, and the Spearman-Brown prediction formula. The computation of the intraclass correlation coefficient via 2-way ANOVA is not encouraged. Several uses and misuses of reliability coefficients applied to dichotomous data are illustrated. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Global shape cannot be attended without object identification.

Tested whether the global shape of objects can be processed without accessing semantic or identity information. Ss judged which of 2 fragmented forms had the same global shape as a reference stimulus. Matching stimuli could be physically identical, semantically related, or unrelated. The reference stimulus and nonmatching (distractor) form could be semantically related or unrelated. Similarity effects in the related condition were unconfounded with matches nameable and nonnameable forms. For nameable forms, related matching forms facilitated performance; a related distractor disrupted performance. Semantic interference was eliminated when nameable distractors were replaced with nonnameable partners; semantic similarity effects on matching were eliminated with a nonnameable reference stimulus and with inverted targets and distractors. Access to information concerning global shape does not normally occur without object identification. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Processing information about covariations that cannot be articulated.

Three experiments with 134 undergraduates and 80 recent high school graduates investigated processing of covariation between verbally described psychological characteristics and appearance of a set of stimulus persons. Based on S. Glucksberg and M. McCloskey's (see record 1982-07068-001) 2-stage question-answering model, it was hypothesized that if the information related to the manipulated covariation was processed and registered, it would result in an increase of processing time for questions that might be considered relevant to the covariation. Results indicate that although Ss were unable to articulate the manipulated covariation in any of the experiments, the pattern of response latencies obtained conformed exactly to the predictions. In 2 of these experiments, effects of the stimulus material in Ss' subsequent judgments were found, consistent with the model. Ss behaved as if they had "learned" the rule implied by the covariation and followed it in their subsequent judgments. (30 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Coefficient alpha and related internal consistency reliability coefficients.

The author studied the conditions under which coefficient alpha and 10 related internal consistency reliability coefficients underestimate the reliability of a measure. Simulated data showed that alpha, though reasonably robust when computed on n components in moderately heterogeneous data, can under certain conditions seriously underestimate the reliability of a measure. Consequently, alpha, when used in corrections for attenuation, can result in nontrivial overestimation of the corrected correlation. Most of the coefficients studied, including lambda2, did not improve the estimate to any great extent when the data were heterogeneous. The exceptions were stratified alpha and maximal reliability, which performed well when the components were grouped into two subsets, each measuring a different factor, and maximized lambda4, which provided the most consistently accurate estimate of the reliability in all simulations studied. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Large-sample confidence intervals for validity and reliability coefficients.

Large-sample confidence intervals (CI) for reliability, validity, and unattenuated validity are presented. The CI for unattenuated validity is based on the Bonferroni inequality, which relies on one CI for test–retest reliability and one for validity. Covered are four reliability–validity situations: (a) both estimates were from random samples; (b) reliability was from a random sample but validity was from a selected sample; (c) validity was from a random sample but reliability was from a selected sample; and (d) both estimates were from selected samples. All CIs were evaluated by using a simulation. CIs on reliability, validity, or unattenuated validity are accurate as long as selection ratio is at least 20% and selected sample size is 100 or larger. When selection ratio is less than 20%, estimators tend to underestimate their parameters. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Let's be realistic: When grief counseling is effective and when it's not.

Comments on the article by D. Larson and W. Hoyt (see record 2007-11559-003) which argued that bereavement researchers have erroneously and unscientifically advocated the pessimistic conclusion that grief counseling is ineffective and perhaps even harmful. They proclaimed that the news is actually good: Grief counseling is not harmful but is as effective as other forms of psychotherapy. Therefore, they concluded, most or all bereaved people should be considered candidates for treatment. This kind of unwarranted optimism is as dangerous, if not more so, than an overly cautious pessimism. The current authors contend that the debate should be focused on (a) fine tuning assessment instruments so that they can best identify those bereaved people in serious clinical need and (b) further developing effective treatments that can more judiciously intervene when intervention is called for. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Exploratory study of individual assessment practices: Interrater reliability and judgments of assessor effectiveness.

Three psychologists who regularly conduct individual assessments were asked to assess 3 individuals posing as job candidates for the same position. The materials from these 9 assessments—test scores, biographical information, and audiotapes of interviews—served as protocols for 50 industrial/organizational psychologists who rated the candidates and assessors. Comparisons of the approaches and conclusions of the assessors indicated variability in job/organizational information obtained, test instruments used, personal history information gathered, interview and the report generated, and conclusions regarding the candidates. On average, only one third of the raters agreed with the conclusions of the assessor whose protocol they were reviewing. Significant differences were found in the raters' evaluations of 2 of the assessors, depending on which candidate the assessor had evaluated. The study's design limits generalizability; however, the low interrater agreement is disturbing. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Group and personal attachments: Two is better than one when predicting college adjustment.

The current study applied Smith, Murphy, and Coates' (1999) group attachment measure to college adjustment using 109 college students. Prior researchers have found that adult dyadic attachment styles predicted college adjustment. This article is the first to explore the relationship between both group and dyadic attachment styles and college adjustment as measured by the Student Adaptation to College Questionnaire. Hierarchical regression analysis revealed that personal attachment anxiety, not avoidance, accounted for the most variance in college adjustment. Group attachment avoidance also accounted for a significant amount of variance, above and beyond dyadic attachment styles, in the prediction of college adjustment. This study supports the importance of exploring both dyadic and group attachment styles in studying overall adjustment to the transition to college life. Implications of the findings for research and practice are discussed. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Nonspecific effects of exposure on stimuli that cannot be recognized.

The repeated exposure of unmasked irregular geometric shapes for very brief durations (1 or 2 ms) has been shown to generate preferences as well as judgments of familiarity for the previously exposed shapes. At the same time these stimuli are not recognized as having been presented. Such exposure also leads to judgments of brightness and darkness independent of stimulus intensity, and it is dependent on the use of unmasked stimuli. This effect is nonspecific, in contrast to stimulus-specific effects with masked stimuli, and it is not restricted to affective judgments. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Log-linear approaches to lag-sequential analysis when consecutive codes may and cannot repeat.

G. P. Sackett's (1979) lag-sequential analysis is recast in log-linear terms. Differing from articles by P. D. Allison and J. K. Liker (see record 1982-12033-001) and by D. Iacobucci and S. Wasserman (see record 1989-14205-001), sequences that, for logical reasons, do not allow consecutive events to be assigned the same code are considered. In addition, overlapped sampling, which the sequential analysis literature usually assumes, is contrasted with nonoverlapped sampling of sequences, which is also used and may seem more faithful to some statistical models. Several advantages of a log-linear approach to sequential problems are noted, including its ability to deal routinely with the structural zeros created when consecutive codes cannot repeat and its integration of sequential methods into an established and well supported statistical tradition. (PsycINFO Database Record (c) 2010 APA, all rights reserved)