Receiver operating characteristic curves for analysis of the results of cervicovaginal smears. A useful quality improvement tool

OBJECTIVES: To assess the value of receiver operating characteristic curves (ROCs) as a tool for improvement in the evaluation of cervicovaginal smear results. DESIGN: We reviewed the results of cervicovaginal smear interpretation for 1 year by 10 different cytopathologists working in the same laboratory (total number of cases, 45,356) and compared them with results of histologic evaluation of corresponding biopsies (n = 2090, 4.6%), the latter taken to be the gold standard. Receiver operating characteristic curves were generated for the laboratory as a whole and for individual cytopathologists. RESULTS: Analysis of these receiver operating characteristic curves was surprisingly informative. They showed that cytopathologists with different diagnostic thresholds could be distinguished from each other. A difference in diagnostic threshold could be distinguished from a difference in diagnostic accuracy. CONCLUSIONS: We conclude that while receiver operating characteristic curves for cervicovaginal smear interpretation have limitations, the results can be used for quality improvement.