Interrater agreement and predictive validity.

Ss were 549 illiterate Iranian truck drivers rated for intelligence and given an individual intelligence test. To relate unreliability of ratings to validity, correlations were made between intelligence test scores and 4 groups of criterion ratings differing in reliability. The authors conclude that in the construction of rating scales, weighting of ratings by their agreement is better than by their disagreement. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interrater reliability and agreement of subjective judgments.

Indexes of interrater reliability and agreement are reviewed, and suggestions are made regarding their use in counseling psychology research. The distinction between agreement and reliability is clarified, and the relationships between these indexes and the level of measurement and type of replication are discussed. Indexes of interrater reliability appropriate for use with ordinal and interval scales are considered. The intraclass correlation as a measure of interrater reliability is discussed in terms of the treatment of between-raters variance and the appropriateness of reliability estimates based on composite or individual ratings. The advisability of optimal weighting schemes for calculating composite ratings is also considered. Measures of interrater agreement for ordinal and interval scales are described, as are measures of interrater agreement for data at the nominal level of measurement. (54 ref) (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interrater agreement in dimensionality and combination of assessment center judgments.

The present study represents an attempt to examine the interrater reliability of assessment center ratings, the dimensionality of those ratings, the consistency of that dimensionality across raters, and the rater's combination of information to arrive at an overall rating. Before- and after-discussion ratings by 4 assessors of 101 potential middle managers on 17 judgmental dimensions were intercorrelated and subjected to principal-components and regression analyses. Results indicate high interrater reliability along with a great degree of similarity across raters in the perceived dimensionality of ratings. Three dimensions—administrative skills, interpersonal skills, and activity level—were the major determinants of overall ratings for all 4 raters. Test scores did not greatly affect overall ratings. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Epistemic progress and journal page limitations: A proposal for increasing the base rate of manuscript acceptance in psychology journals.

Proposes that the requirement of publication and submission fees would (1) minimize the amount of extraneous information and speculation in journal articles and (2) encourage the use of brief (2–3 page) reports. Also, additional pages could be allocated for psychological research, and researchers would be able to keep abreast of a greater number of findings and ideas in less time, more efficiently, and without additional financial burdens for journal publishers or subscribers. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interrater agreement and stability of the Functional Independence Measure for Children (WeeFIM): use in children with developmental disabilities

OBJECTIVE: Examination of the interrater agreement and stability of ratings obtained using the Functional Independence Measure for Children (WeeFIM) in a sample of children with developmental disabilities. DESIGN: A relational design was used in which two sets of WeeFIM scores were collected under four conditions: same rater-short interval; same rater-long interval; different rater-short interval; and different rater-long interval. SETTING: WeeFIM scores were collected in outpatient developmental rehabilitation centers, school programs, and the children's homes. PARTICIPANTS: Data were collected for 205 children ranging in age from 11 to 87 months. All children had a medical diagnosis of disability and were receiving habilitative-educational intervention or follow-along services including neurodevelopmental surveillance. INSTRUMENT: The WeeFIM instrument examines basic daily living and functional skills in children from birth to 7 years of age. The WeeFIM is modeled after the Functional Independence Measure (FIM) for adults and includes 18 items in the following subscales: self-care, sphincter control, transfers, locomotion, communication, and social cognition. RESULTS: Kappa values for items ranged from .44 to .82. Intraclass correlation coefficients (ICC) for the six subscales ranged from .73 to .98. Total WeeFIM ICC values were greater than .95 for all analyses. CONCLUSIONS: The WeeFIM ratings for the 205 children with developmental disabilities participating in this investigation were consistent across raters and time.     

On manuscript evaluation procedures.

Comments that the article by D. D. Bowen et al (see record 1973-11147-001) expresses dissatisfaction with the reliability and objectivity of referees. Other studies (e.g., J. Bernard, 1964) suggest that women are considered less seriously than men in intellectual realms. The author proposes evaluation procedures that assure anonymity of author and institution as one way of avoiding bias, and suggests that the American Psychological Association (APA) revise its bylaws to make this the policy for all APA journals. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A journal for psychological essays.

Supports W. Webb's (1965) proposal for a journal for the publication of psychological essays, arguing that there is an overemphasis on empirical studies and a neglect of speculative essays in current psychology. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

A new vision for the journal.

It is with great pleasure that I accept and begin my tenure as the fourth editor of the International Journal of Stress Management (IJSM), following F. J. McGuigan, John Carlson, and Gary VandenBos. In this brief editorial, I provide historical context to set the path for the future of the journal and reflect on two parts of the journal’s title, “international” and “management,” that shape my vision for the journal. My envisioned future for the journal is increased focus on international and cross-cultural perspectives, as well as an interdisciplinary approach to gain a comprehensive understanding of factors that shape (a) individuals’ perceptions and experiences of stimuli (or stressors); (b) the environment’s internal and external pressures; (c) individuals’ perceptions of physiological, psychological, and behavioral health and well-being; as well as (d) individuals’ and organizations’ efforts toward prevention, coping, and management of healthy lifestyles. My ultimate goal is to see IJSM serve as the desired publication outlet of choice and primary journal for scholars and practitioners around the globe who are focused on the progress of scholarship and practice in identification and management of stress. (PsycINFO Database Record (c) 2011 APA, all rights reserved)     

Measuring diagnostic agreement.

Diagnostic agreement tests the reliability and concordance of diagnostic systems. The introduction of measures of agreement with reputations for baserate independence (e.g., Yule's Y and Q), and new studies occasioned by the publication of the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; American Psychiatric Association, 1994) and the International Classification of Diseases—10 (ICD-10, World Health Organization, 1992) make it necessary to study the relationship of illness baserates to measures of agreement. Testing diagnostic concordance for diagnoses of drug dependence from the third edition of the DSM (American Psychiatric Association, 1980) versus DSM-IV diagnoses of drug dependence under 3 baserate conditions, it was found that Yule's Y and Q proved as vulnerable to differences in baserates as kappa or percent agreement and that specificity covaried with baserate rather than being fixed, as most theoretical discussions assume. The uncritical use of Y and Q, therefore, is likely to lead to optimistic interpretations of agreement. Kappa should be preferred for most purposes, although an adjustment to the computational formulas for Y and Q is presented that can diminish their positive bias. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interobserver relational agreement.

Assessing interobserver agreement calls for measuring the degree to which numbers generated by one observer match those generated by another observer. However, for all scales of measurement save one, the absolute scale, using interobserver agreement as a measure of interobserver consistency is too strict because the observers might disagree only on empirically meaningless relationships. Two observers that are rating behaviors on an ordinal scale need only to generate orders that agree, not ratings that agree. This concept is formalized into a notion of relational agreement. Observers need to agree only on empirically meaningful (in a measurement theoretical sense) relationships. Those relationships that are empirically meaningful change as a function of the scale of measurement in use. A class of measures for measuring relational agreement (based on F. E. Zegers and J. M. F. ten Berge [see PA, Vol 72:24356]) is presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Summary report of journal operations for 1978.

Provides an American Psychological Association journal summary compiled from the 1978 annual reports of the Council of Editors and from Central Office records. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Summary report of journal operations for 1974.

This summary is compiled from the 1974 annual reports of the Council of Editors and from Central Office business records. The corresponding figures for 1973 can be found in the June 1974 issue of the American Psychologist. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Summary report of journal operations for 1975.

The following summary is compiled from the 1975 annual reports of the Council of Editors and from Central Office records. The corresponding figures for 1974 can be found in the June 1975 issue of the American Psychologist. (PsycINFO Database Record (c) 2010 APA, all rights reserved)