Correcting the bias of clerkship timing on academic performance

OBJECTIVES: To document the time-of-year bias in National Board of Medical Examiners subject examination (NBME) scores in a third-year pediatrics clerkship and to develop a grading method that neutralizes the bias. DESIGN: Interventional modeling of final grades. SETTING: University-based medical school. SUBJECTS AND METHODS: During each of the past 3 academic years, we conducted six 2-month pediatric clerkships for third-year students. To counter the time-of-year bias, NBME scores, clinical evaluations, and departmental examination scores for the current rotation were pooled with those from the rotations from the same time of year during the previous 2 years. Final grades for the current rotation were determined by cutoff points derived from that entire 3-year pool. We analyzed this approach by testing the time-of-year effects on NBME scores, clinical evaluations, and final grades while maintaining step 1 of the US Medical Licensing Examination as a preclinical baseline control. RESULTS: The scores for step 1 of the US Medical Licensing Examination did not differ significantly by time of year. Clinical evaluations and NBME scores showed significant upward trends as the academic year progressed. By contrast, according to design, final grades showed no significant time-of-year trend. CONCLUSIONS: Our results support previous reports of significant improvements in NBME scores across the academic year. Our method of computing final grades corrects for this time-of-year bias by judging students only in relation to those who took the rotation at the same time of year. It is our belief that the prevalence and significance of the time-of-year trend warrants such an adjustment in grading to help minimize the academic disadvantage faced by students early in their clinical training.