Ensuring reliability of outcome measures in multicenter clinical trials of treatments for acute ischemic stroke. The program developed for the Trial of Org 10172 in Acute Stroke Treatment (TOAST)

BACKGROUND AND PURPOSE: Ensuring the reliability and validity of outcome measures used in clinical trials is essential to the success of the trial. The Trial of Org 10172 in Acute Stroke Treatment (TOAST) is a multicenter clinical trial that is recruiting patients with acute ischemic stroke seen at medical centers across the United States. METHODS: This paper describes an approach to train physicians to use three clinical measures: the National Institutes of Health (NIH) Stroke Scale, a supplemental motor examination, and the Glasgow Outcome Scale. The program included education, certification, remediation when needed, monitoring, and reliability assessment. The goal was to ensure that interrater assessments were as equivalent to one another as possible. RESULTS: Of the first 95 clinicians who began the certification process, 75 passed during the first evaluation. Eighteen of the other physicians were able to complete the process after remediation. The intraclass correlations of both the NIH Stroke Scale and supplemental motor examination exceeded 0.95. The kappa values for the Glasgow Outcome Scale were 0.61 and 0.62 for the first and second ratings of the videotape, respectively. CONCLUSIONS: Our experience suggests that a program that includes educational and certification processes can be performed as part of the design of a multicenter clinical trial. The method of providing educational and testing videotapes to each site so that physicians can be trained and certified is an effective, inexpensive, and practical approach for enhancing and certifying the expertise of the large number of physicians involved in a multicenter study.