Observer variability among colposcopists from the West Midlands region

OBJECTIVE: To assess variation in diagnoses and management decisions among colposcopists when presented with cervical images; to see the impact of the referral cytology report on diagnostic accuracy. DESIGN: A two-part video questionnaire study. PARTICIPANTS: Colposcopists from West Midlands Region. METHODS: Twenty cervical images displaying a range of transformation zones from normal through varying abnormalities up to cervical intraepithelial neoplasia (CIN) grade 3 were shown on video tape together with basic patient information. Two sets of videos were made, the second being identical to the first other than including the referral cytology. Participants recorded their diagnoses and management decisions on prepared questionnaires. The two sets of videos were viewed several weeks apart. RESULTS: Completed questionnaires to both videos were received from 30 colposcopists. Diagnostic accuracy improved with knowledge of the cervical cytology result in cases of CIN 2/3 (chi 2 = 19.45, P < 0.0001) but not where the histology was CIN 1 or less (chi 2 = 2.64, P = 0.10). Overall interobserver agreement improved slightly from kappa = 0.169 to kappa = 0.212 when the cytology was revealed. While only 2.6% of cases of CIN 2/3 would have been under-managed after the second questionnaire, 37.5% cases where the abnormality did not amount to CIN would have been overtreated. CONCLUSION: There is considerable inter-observer variability and variation in diagnostic accuracy in scoring cervical images particularly at the lower end of the spectrum of abnormality which has the potential to lead to over-treatment. We rely considerably on the cervical cytology result in forming a diagnosis. We recommend that a see-and-treat approach be abandoned when the referral smear shows minor abnormalities. The study has implications for both training and audit in colposcopy.