| Abstract: | Assessing interobserver agreement calls for measuring the degree to which numbers generated by one observer match those generated by another observer. However, for all scales of measurement save one, the absolute scale, using interobserver agreement as a measure of interobserver consistency is too strict because the observers might disagree only on empirically meaningless relationships. Two observers that are rating behaviors on an ordinal scale need only to generate orders that agree, not ratings that agree. This concept is formalized into a notion of relational agreement. Observers need to agree only on empirically meaningful (in a measurement theoretical sense) relationships. Those relationships that are empirically meaningful change as a function of the scale of measurement in use. A class of measures for measuring relational agreement (based on F. E. Zegers and J. M. F. ten Berge [see PA, Vol 72:24356]) is presented. (PsycINFO Database Record (c) 2010 APA, all rights reserved) |
