Reliability of the Part II board certification examination in psychiatry: examination stability

OBJECTIVE: The aim of this study was to examine the reliability (examination stability) of the American Board of Psychiatry and Neurology (ABPN) Part II (oral) examination in psychiatry. METHOD: The authors analyzed the consistency (agreement between grades given by two independent examiners) for a 1-year examination cycle using a weighted kappa statistic and compared different parts of the examination (live patient and videotape), different examination sites, different days, and different times of the day. RESULTS: There was no significant difference in agreement between examiners by different parts of the examination, examination site, day of the week, or time of day. CONCLUSIONS: The stability of the Part II ABPN examination in psychiatry is not influenced significantly by the format or site of administration. Candidate performance is the predominant factor in the determination of passing or failing grades.