Accuracy of helical CT for acute pulmonary embolism: ROC analysis of observer performance related to clinical experience

The aim of this study was to test the influence of observer experience on the accuracy for interpreting helical CT for acute pulmonary embolism (PE) and to identify sources of observer errors. Three observers of different expertise blindly assessed 147 helical CT scans for suspected PE (true status regarding absence or presence of PE known from independent reference studies). These observers were (a) an experienced CT radiologist, (b) a fellow in CT, and (c) a second-year resident without any formal training in CT. None of them had prior experience with CT for PE. Firstly, 70 CT scans were scored without revealing true PE status. Afterwards, feedback was provided and another 77 CT scans were evaluated. The CT scans were scored on a 5-point confidence scale and receiver-operator-characteristic analysis was performed. Different sources of interpretation errors were analyzed. The two observers with CT experience were significantly more accurate than the unexperienced observer. Their performance was not influenced by feedback training. Certain observer errors were identified, but there was no clear difference among the three observers considering the type of errors. There is significant influence of observer experience on accuracy of reading helical CT for PE: A basic working experience with whole-body CT seems to be a prerequisite. These results suggest that with this experience any radiologist should be able to achieve good accuracy; helical CT thus might become a suitable technique for acute PE in routine clinical practice.