Simultaneous palpation of the craniosacral rate at the head and feet: intrarater and interrater reliability and rate comparisons

BACKGROUND AND PURPOSE: The main purpose of this study was to determine the interrater and intrarater reliability of measurements obtained during palpation of the craniosacral rate at the head and feet. Palpated craniosacral rates of head and feet measured simultaneously were also compared. Subjects. Twenty-eight adult subjects and 2 craniosacral examiners participated in the study. METHODS: A within-subjects repeated-measures design was used. A standard cubicle privacy curtain, hung over the subject's waist, was used to prevent the examiners from seeing each other. RESULTS: Interrater intraclass correlation coefficients (ICCs) were .08 at the head and .19 at the feet. Intrarater ICCs ranged from .18 to .30. Craniosacral rates simultaneously palpated at the head and feet were different. CONCLUSION AND DISCUSSION: The results did not support the theories that underlie craniosacral therapy or claims that craniosacral motion can be palpated reliably.     

Reliability assessment of isometric knee extension measurements with a computer-assisted hand-held dynamometer

OBJECTIVE: To assess clinically relevant indexes of measurement error of hand-held dynamometer measurements using a computer-assisted hand-held dynamometer (CAHN-DY). DESIGN: In separate sessions, different physical therapists performed repeated measurements of maximal isometric knee extension. SETTING: Four physical therapy practices and outpatient departments. PATIENTS: Consecutive samples of 30 male and 28 female patients aged 13 to 77 years, with isolated orthopedic knee disorders participated in this study. MAIN OUTCOME MEASURES: For intrarater and interrater applications, the standard error of measurement (SEM) and related 95% confidence intervals and minimal detectable changes were assessed. RESULTS: Sixty percent of the patients performed within the therapist's upper force limit. Variance between sessions performed by the same or different therapists accounted for a large percentage of the measurement error (70% to 81%). For single measurements, the standard error of measurement was assessed not to exceed 10N x m. Minimal detectable changes for different designs varied from 13.8 to 28.2N x m. CONCLUSIONS: The CAHN-DY facilitated standardization of test performance in a satisfactory manner, resulting in less measurement error compared with simple hand-held dynamometry.     

Reliability of scores on the Stroke Rehabilitation Assessment of Movement (STREAM) measure

BACKGROUND AND PURPOSE: The Stroke Rehabilitation Assessment of Movement (STREAM) is a new clinical measurement tool for evaluating the recovery of voluntary movement and basic mobility following stroke. This article presents the results of 3 substudies examining the reliability (interrater and intrarater) and internal consistency of STREAM scores. SUBJECTS AND METHODS: A "direct-observation reliability study" was conducted on 20 patients who had strokes and were in a rehabilitation setting. Pairs of raters from a group of 6 participating therapists provided data to judge interrater agreement. A "videotaped assessments reliability study" was done to assess intrarater and interrater agreement on the scoring of videotaped performances using the STREAM measure and involved 4 videotaped assessments that were viewed and rated on 2 occasions by 20 physical therapists. The internal consistency of the STREAM scores was evaluated for 26 patients who had strokes and who demonstrated the full range of motor ability. RESULTS: The reliability of the STREAM scores was demonstrated by generalizability correlation coefficients of .99 for total scores and of .96 to .99 for subscale scores. The internal consistency of the STREAM scores was demonstrated by Cronbach alphas of greater than .98 on the subscales and overall. CONCLUSION AND DISCUSSION: These high levels of reliability support the use of the STREAM instrument for the measurement of motor recovery following stroke. Further work on the validity and responsiveness of the STREAM measure is in progress.     

Comparative analysis of the reliability of job performance ratings.

This study used meta-analytic methods to compare the interrater and intrarater reliabilities of ratings of 10 dimensions of job performance used in the literature; ratings of overall job performance were also examined. There was mixed support for the notion that some dimensions are rated more reliably than others. Supervisory ratings appear to have higher interrater reliability than peer ratings. Consistent with H. R. Rothstein (1990), mean interrater reliability of supervisory ratings of overall job performance was found to be .52. In all cases, interrater reliability is lower than intrarater reliability, indicating that the inappropriate use of intrarater reliability estimates to correct for biases from measurement error leads to biased research results. These findings have important implications for both research and practice. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

Interrater and intrarater reliability in the measurement of kyphosis in postmenopausal women with osteoporosis

STUDY DESIGN: A reliability study was performed using repeated random measurements involving three observers, 26 subjects and three instruments. OBJECTIVES: To determine the most reliable, cost-effective, noninvasive, and clinically feasible method of measuring spinal kyphosis. SUMMARY OF BACKGROUND DATA: The most clinically useful, noninvasive and reliable method of measuring postural deformity in spinal osteoporosis (kyphosis) remains unqualified. Despite traditional use of costly, invasive roentgenographs for the evaluation of spinal kyphosis, the reliability of this method remains questionable. METHODS: Twenty-six postmenopausal women with known bone mineral density and a diagnosis of osteoporosis were recruited from the Osteoporosis Program at Women's College Hospital, Toronto, Canada. Non-invasive measurements of thoracic kyphosis were obtained by three trained examiners using the DeBrunner's kyphometer and the flexicurve ruler. The intrarater and interrater reliability of and between each method was compared, using roentgenographic films obtained in the sagittal plane. Spinal posture was classified according to the method of Itoi (1990). Statistical computations were performed using SAS statistical software. RESULTS: Consistent measurements were obtained with the DeBrunner's kyphometer and the flexicurve ruler by each observer, according to the results of critical two-way analysis of variance (Intraclass Correlation Coefficient 2, 1). Measurements in two subgroups, healthy backs (n = 11) and rounded backs (n = 13), showed consistent use of each noninvasive instrument with some examiner preference for specific tools. There was marginally better intrarater and interrater reliability using the DeBrunner's kyphometer compared with that obtained with the flexicurve ruler. Two-way analysis of variance (Intraclass Correlation Coefficient 2, 1) of collapsed data showed no significant difference in the reliability of the kyphometer, flexicurve ruler, or roentgenographs in the measurement of thoracic kyphosis. CONCLUSIONS: The flexicurve ruler and DeBrunner's kyphometer had the closest agreement in the measurement of spinal kyphosis. The kyphometer demonstrated the least variation in intrarater and interrater reliability when compared with the flexicurve ruler and roentgenographs. The flexicurve ruler permits qualitative assessment of posture, however, and is the most cost-effective instrument. The results of this study challenge the traditional belief that roentgenographic analysis is the best method for evaluating spinal kyphosis. The DeBrunner's kyphometer and flexible ruler may represent viable, cost-effective and noninvasive alternatives to roentgenographic evaluation of spinal kyphosis.     

The modified Gait Abnormality Rating Scale for recognizing the risk of recurrent falls in community-dwelling elderly adults

BACKGROUND AND PURPOSE: The purpose of this study was to determine the reliability and validity of measurements obtained with a seven-item modified version of the Gait Abnormality Rating Scale (GARS-M), an assessment of gait designed to predict risk of falling among community-dwelling, frail older persons. SUBJECTS: Fifty-two community-dwelling, frail older persons, with a mean age of 74.8 years (SD = 6.75), participated. METHODS: A history of falls was determined from self-report or by proxy report. The GARS-M was scored from videotapes of subjects walking at self-selected paces. Gait characteristics were recorded during a timed walk on a 6-m brown-paper walkway. RESULTS: Scores obtained by three raters for 23 subjects demonstrated moderate to substantial intrarater and interrater reliability. Concurrent validity, as assessed by Spearman rank-order correlation coefficients, was demonstrated for the relationship between GARS-M scores and stride length (r = -.754) and for the relationship between GARS-M scores and walking speed (r = -.679). Mean GARS-M scores distinguished between frail older persons with and without a history of recurrent falls (mean GARS-M scores of 9.0 and 3.8, respectively). CONCLUSION AND DISCUSSION: The GARS-M is a reliable and valid measure for documenting gait features associated with an increased risk of falling among community-dwelling, frail older persons and may provide a clinically useful alternative to established quantitative gait-assessment methods.     

Interrater reliability of six tests of trunk muscle function and endurance

Some studies have shown a relationship between trunk muscle strength and low back pain. Measures of trunk muscle strength and endurance, which are feasible in the clinical setting, are needed. The purpose of this study was to determine interrater reliability of six tests of abdominal and trunk extensor muscle strength and endurance. The tests included abdominal and extensor dynamic endurance, hand-held dynamometry of isometric flexion and extension, and abdominal and extensor static endurance. Thirty-nine healthy workers were recruited as subjects. Each was tested by three raters on 3 days within 1 week. Intraclass correlation coefficients (ICC) and the standard error of measurement (SEM) were calculated: abdominal dynamic endurance ICC = .89, SEM = 8 repetitions; extensor dynamic endurance ICC = .78, SEM = 9 repetitions; abdominal isometric force ICC = .25, SEM = 60 N; extensor isometric force ICC = .24, SEM = 68 N; abdominal static endurance ICC = .51, SEM = 35 seconds; extensor static endurance ICC = .59, SEM = 20 seconds. The dynamic endurance tests had acceptable interrater reliability. For the others, reliability was poor and the SEMs were large.     

Agreement and differences between reviewers in a peer review procedure

In Germany the statutory pension insurance institutions have started a quality assurance programme. Our institute developed a peer review procedure for screening the process quality of rehabilitation care. The peer review was tested in a pilot study. Our article refers to the examination of interrater reliability, intrarater reliability and reviewer bias. First of all, experienced doctors were trained in reviewing reports routinely written by rehabilitation doctors at discharge of their patients. The peers had to judge on 56 process criteria belonging to six categories (e.g. case history). The reliability coefficients were calculated for the overall judgement of each category and the overall judgement of the process quality of rehabilitation care. The coefficients of interrater reliability and the coefficients of average intrarater reliability range from sufficient to good. Only few reviewers showed a general tendency to harsh or lenient rating. The objectivity of the tested peer review procedure seems definitely higher than in American studies of peer review of hospital charts.     

Induction chemotherapy, surgery, and concomitant chemoradiotherapy for carcinoma of the esophagus: a long-term analysis

OBJECTIVE: To examine intrarater reliability in measurements of active range of motion and passive range of motion of shoulder flexion and abduction when motions are assessed in sitting, as compared with supine. DESIGN: Thirty adult subjects were measured eight times, in random order, for each of the two shoulder motions: two passive and two active measurements while sitting, and two passive and two active measurements while supine. Data were analyzed to determine intraclass correlation coefficients (ICCs) and paired t values between trials 1 and 2 for measurements in the same position, and between sitting and supine trials for each type of measurement. SETTINGS: Rehabilitation facility and university. STUDY POPULATION: Volunteer sample: 11 rehabilitation inpatients; 19 university students. RESULTS: ICCs between trials 1 and 2 on comparable measurements in the same position indicated high intrarater reliability for active and passive measurements, regardless of testing position. ICCs between comparable measurements in the two testing positions indicated only a moderate level of agreement. Paired t tests between comparable readings taken in sitting versus supine revealed no significant differences for flexion, but significantly higher measurements of abduction when testing in the supine position. CONCLUSIONS: Measurements in sitting or supine yield similarly high intrarater reliability. Lowered reliability between measurements taken in different positions indicates that test position should be routinely recorded, and repeated clinical measures of individual subjects should be administered in a consistent position.     

Intrasubject reliability of spinal range of motion and velocity determined by video motion analysis

BACKGROUND AND PURPOSE: The purpose of this study was to investigate the repeatability of spinal range of motion (ROM) and movement velocity measurements of patients with chronic low back pain, using a two-dimensional motion analysis system. This apparatus uses reflective markers placed on anatomical landmarks and video digitization to derive ROM measurements from three segments of the spine and associated velocities through the respective ROMs. SUBJECTS: Forty-two patients with chronic LBP underwent ROM and movement velocity testing. METHODS: Each subject was tested twice without removal of the markers to minimize error contribution from differences in marker placement. RESULTS: Results indicated that both the ROM measures and the velocity measures were highly repeatable. Intraclass correlations for the ROM measures ranged from .77 to .96. Velocity measures were also reliable, with intraclass correlation coefficients ranging from .75 to .97. CONCLUSION AND DISCUSSION: Overall, the results seem to indicate that the video motion analysis system used in this study yields repeatable ROM and velocity measures on a clinical population. In practice, however, the measures may reflect greater errors due to the need of examiners to relocate markers at different testing sessions. These systems also offer distinct advantages over other means of obtaining ROM and velocity measures. The results of this study indicate that these measures may be obtained without undue concern for measurement artifact due to the instrumentation reliability.     

Quality of life in elderly patients with COPD: measurement and predictive factors

OBJECTIVE: To examine the interrater reliability and completion time of the Functional Assessment Measure, which is the Functional Independence Measure (FIM) plus additional items (FIM+FAM). DESIGN: Interrater reliability study. SETTING: Inpatient rehabilitation units of a postacute care brain injury rehabilitation program. PATIENTS: A convenience sample of 53 extremely severely impaired adult survivors of traumatic brain injuries (40 men, 13 women, mean age 38yrs). MAIN OUTCOME MEASURES: Treatment team members' ratings of the 30 FIM +FAM items, and time taken to complete the FIM+FAM. RESULTS: Intraclass Correlation Coefficients (ICCs) were within the good to excellent range (ICC > .60) for 29 of 30 items and for all subscales except psychosocial adjustment. Higher mean ICC values were obtained for motor domain items than for cognitive/psychosocial domain items. Treatment teams became progressively faster over a 12-week period in completing the FIM+FAM. The generally good to excellent range interrater reliability found in this study helps support the use of the FIM+FAM in rehabilitation settings. Further support was obtained for the finding that motor items are more reliable than cognitive and psychosocial items. Administration of the FIM+FAM can be done in a timely manner in a rehabilitation setting.     

Ophthalmoscopic measurement of the optic disc

PURPOSE: The authors evaluated whether the optic disc dimensions can be measured directly by ophthalmoscopy without the use of a fundus camera or special ophthalmoscope. MATERIALS AND METHODS: The horizontal and vertical disc diameters were measured ophthalmoscopically in 158 eyes of 95 subjects using a standard Goldmann three-mirror contact lens and a commercial slit lamp with adjustable length of the beam. The refractive error ranged from -7.25 to 3.5 diopters. Based on these direct measurements, the authors calculated the disc area by applying a modified formula for an ellipse, where area = pi/4 x the horizontal diameter x the vertical diameter. Additionally, the authors determined by planimetry the diameters and area of the disc on color photographs after correcting the ocular and photographic magnification. RESULTS: The values of the disc diameters as determined by photography were x1.27 larger than those measured with the Goldmann contact lens. Taking into account this constant correction factor of 1.27 for all eyes examined, the disc diameters, as measured by the Goldmann lens, varied by 0.11 +/- 0.06 mm or 5.8% +/- 3.5% from the values measured on the photographs. This error decreased slightly (P = 0.04) with increasing disc size. It was independent of the refractive error. The calculated disc area based on the ophthalmoscopically determined disc diameters differed by 9.4% +/- 6.6% from the disc area as measured by planimetry on the photographs. CONCLUSIONS: For clinical purposes, the optic disc and other structures of the posterior fundus can be measured by ophthalmoscopy using a Goldmann contact lens and a slit lamp with adjustable beam length. The optic disc area can be calculated by the formula: horizontal disc diameter x vertical disc diameter x pi/4.     

Antimyoclonic effect of gabapentin in a posthypoxic animal model of myoclonus

The perceptual GRBAS scale for deviant voice quality (completed with a "I" parameter: Instability = fluctuation of voice quality over time) was tested in 5 different institutes on 943 voice patients. Each voice was evaluated separately by 2 professionals. The interrater correlation reaches 0.87 for G (grade), 0.70 for R (roughness), 0.69 for B (breathiness) and 0.65 for A/S (asthenicity/strain). Experience with the scale significantly improves the interrater agreement. The intrarater correlation (80 voices; time interval 2-6 months) is systematically slightly stronger than the interrater correlation. Further, GIRBAS/acoustical correlations were investigated on 80 pathological voices, by using the Multidimensional Voice Program (MDVP) of Kay. Principal component analysis allows reduction of 21 MDVP parameters to 5 clinically relevant aspects, with a typical acoustical measurement for each of them. The strongest correlations are found between G and Shimmer/Noise to harmonics ratio, R and Jitter, and B and Shimmer. As tremor and diplophonia are easily identified perceptually and acoustically, it seems useful to add this information respectively to AS and R. The GIRBAS scale seems to be a valuable instrument for clinical practice.     

Inhibition of triazolam clearance by macrolide antimicrobial agents: in vitro correlates and dynamic consequences

OBJECTIVE: To evaluate the ability of the DSM-IV criteria for the pervasive developmental disorders (PDD) to reliably and accurately differentiate PDD subtypes. METHOD: The sample consisted of 143 children with various types of developmental disabilities. A diagnosis of PDD and PDD subtype was made by one clinician using information obtained from the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule. The raw data from the Autism Diagnostic Interview-Revised, clinical notes (excluding diagnostic opinion), Autism Diagnostic Observation Schedule, IQ, and other available data were independently assessed by three experienced raters, each of whom then made a separate, blind diagnosis. If there was any disagreement, a consensus best-estimate (CBE) diagnosis was made after discussion. To assess reliability, the agreement between the three raters was calculated using k. Accuracy was assessed by calculating the agreement between the clinician's diagnosis and the CBE and by calculating the error rates associated with the three raters using latent class analysis. RESULTS: The current DSM-IV criteria show good to excellent reliability for the diagnosis of PDD, Asperger's disorder (AsD), and autism, but they show poor reliability for the diagnosis of atypical autism. The clinician (compared to the CBE) had little difficulty differentiating PDD from non-PDD children and autism from AsD but had more difficulty identifying children with atypical autism. The latent class analysis also showed that the average error rates of the three raters for a differentiation of atypical autism from autism were unacceptably high. CONCLUSIONS: Although the psychometric properties of the current DSM-IV criteria for autism and AsD appear quite acceptable, there is likely to be a high rate of misclassification of children given a diagnosis of atypical autism.     

Immunologic and renal markers among photogravure printers exposed to toluene

Despite the severity of ankle syndesmosis injuries, it has been argued that they are relatively poorly detected. This study investigated the interrater reliability of four orthopaedic tests for ankle syndesmosis injury and assessed their ability to predict the extended recovery times that have been reported as a consequence of this injury. Nine physiotherapists, working in random pairings, examined 53 subjects, all receiving treatment for ankle injury at two private clinics. Each subject was tested by two physiotherapists who independently performed the palpation test, external rotation test, squeeze test, and dorsiflexion-compression test. Kappa coefficient testing indicated that the external rotation test had the best interrater reliability (kappa = 0.75). The squeeze test was found to have moderate reliability (kappa = 0.50), and the palpation and dorsiflexion-compression tests both had only fair reliability (kappa = 0.36). The degree of pair-wise association between the results of the four tests was low (phi < or = 0.30 for all test combinations), suggesting that if all four tests were performed on the same subject, it was not likely that they would achieve similar results. Follow-up interviews were conducted to determine the time taken for subjects to walk 10 m without pain and, for sports injuries, the time taken to return to training and then to competition. For each test, Mann-Whitney U values showed no significant difference between the recovery times of subjects with positive or negative test results, although subjects with recovery times markedly longer than normal were detected by three of the four tests. When an either/or combination of the external rotation and dorsiflexion-compression tests was considered, subjects with a positive test result took significantly longer to return to playing sports.     

The Rorschach: Facts, fictions, and future.

A large body of empirical evidence supports the reliability, validity, and utility of the Rorschach. This same evidence reveals that the recent criticisms of the Rorschach are largely without merit. This article systematically addresses several significant Rorschach components: interrater and temporal consistency reliability, normative data and diversity, methodological issues, specific applications in the evaluation of thought disorder and suicide, meta-analyses, incremental validity, clinician judgment, patterns of use, and clinical utility. Strengths and weaknesses of the test are addressed, and research recommendations are made. This information should give the reader both an appreciation for the substantial, but often overlooked, research basis for the Rorschach and an appreciation of the challenges that lie ahead. (PsycINFO Database Record (c) 2010 APA, all rights reserved)