Agreement between administrative files and written medical records: a case of the Department of Veterans Affairs

OBJECTIVES: This study examined the reliability of Department of Veterans Affairs' health information databases concerning patient demographics, use of care, and diagnoses. METHODS: The Department of Veterans Affairs' Patient Treatment files for Main, Bed-section (PTF) and Outpatient Care (OCF) were compared with medical charts and administrative records (MR) for a random national sample of 1,356 outpatient visits and 414 inpatient discharges to Department of Veterans Affairs' facilities between July 1 and September 30, 1995. Records were uniformly abstracted by a focus group of utilization review nurses and medical record coders blinded to administrative file entries. RESULTS: Reliability was adequate for demographics (kappa approximately 0.92), length of stay (agreement=98%), and selected diagnoses (kappa ranged 0.39 to 1.0). Reliability was generally inadequate to identify the treating bedsection or clinic (kappa approximately 0.5). Compared with medical charts, Patient Treatment Files/Outpatient Care Files reported an additional diagnosis per discharge and 0.8 clinic stops per outpatient visit, resulting in higher estimates of disease prevalence (+39% heart disease, +19% diabetes) and outpatient costs (+36% per unique outpatient per quarter). CONCLUSIONS: In the absence of pilot work validating key data elements, investigators are advised to construct health and utilization data from multiple sources. Further validation studies of administrative files should focus on the relation between process of data capture and data validity.