Validation of an objective structured clinical examination in psychiatry

PURPOSE: To examine the validity of a psychiatry clerkship's objective structured clinical examination (OSCE). METHOD: In 1996, 33 clinical clerks and 17 psychiatry residents at the University of Toronto participated in an eight-station OSCE evaluated by psychiatrist-examiners using binary checklists and global ratings. Prior to the OSCE, communication course instructors were asked to rank the clerks on interviewing ability, and faculty supervisors were asked to identify the OSCE stations on which the clerks were likely to do well or poorly. RESULTS: Mean OSCE scores were significantly higher for the residents than for the clerks on global ratings but not on checklists. The communication instructors accurately predicted the clerks' rankings on the global scores but not their scores on the checklists. The faculty supervisors predicted with moderate accuracy the clerks' success on the OSCE stations as measured by the checklists but not by the global ratings. The residents rated the OSCE scenarios as highly realistic. CONCLUSIONS: The evidence of construct and concurrent validity together with high ratings of realism suggest that a psychiatry OSCE can be a valid assessment of clerks' clinical competence.     

Predictive value of letters of recommendation vs questionnaires for emergency medicine resident performance

OBJECTIVE: To evaluate the predictive value of standard letters of recommendation (LORs) vs preprinted questionnaires (PPQs) for resident performance at one emergency medicine (EM) residency program. METHODS: A retrospective association of LORs and PPQs with in-training residents performance ratings was done at one EM residency program. The residency application files of EM residents who completed the program were reviewed to locate files that had LORs and PPQs written by the same author. Seventeen resident files contained 32 LOR/PPQ pairs. These LORs and PPQs were submitted in a blinded fashion to 3 outside EM residency directors. Each LOR and PPQ was evaluated for the applicant's suitability for the specialty of EM, medical knowledge, procedural skills, interpersonal skills, motivation, and overall rank. The scores given by the outside reviewers were compared with resident performance ratings determined by 5 EM attending physicians who evaluated the residents along the same 6 dimensional ratings. RESULTS: Statistically, no differences were found between the LORs and PPQs in predicting resident performance. CONCLUSIONS: PPQs may substitute for LORs in the evaluation of resident applicants.     

Effects of dietary manipulations and glucose infusion on glucagon response during exercise in rats

OBJECTIVE: To describe our initial experience with a computerized telecommunication system, termed the interactive voice-response system, to record resident performance of laparoscopic surgery. METHODS: After completing a laparoscopic procedure, the surgeon and resident telephone a toll-free number independently and respond to three prerecorded statements using a Likert scale of 1 to 5. The caller then is asked to describe the resident's response to critical incidents or elements of surprise that arose during the surgery. The ratings and verbal comments are compiled, transcribed, and forwarded to the respective resident. The resident (and program director) can hear the verbal comments by entering a four-digit code. RESULTS: Between May 1, 1995, and May 31, 1996, 430 cases were reported by 11 surgeons and 16 residents using the interactive voice-response system. One hundred ninety-five (45%) procedures were entered by both the resident and surgeon. A survey undertaken during the introductory phase of the project revealed that five of the seven residents exposed to the system found that it provided useful feedback and preferred the system to traditional in-service reporting methods. In addition, five residents thought that the system complemented the personal feedback they received in the operating room. CONCLUSION: The system has been accepted by both residents and surgeons and has addressed the important components of resident in-training evaluation, namely, evaluation on a case-by-case basis, timely feedback, and self-assessment of resident performance.     

Performance of "standardized examinees" in a standardized-patient examination of clinical skills

PURPOSE: As a first step in testing the utility of using trained "standardized examinees" (SEs) as a quality-assurance measure for the scoring process in a standardized-patient (SP) examination, to test whether medical residents could simulate students in an SP examination and perform consistently to specified levels under test conditions. METHOD: Fourth-year students from the Baltimore-Washington Consortium for SPs participated in a National Board of Medical Examiners Prototype Examination of clinical skills consisting of twelve 15-minute student-patient encounters in 1994-95. For this examination, internal medicine residents were trained to act as ordinary candidates and to achieve target scores by performing to a set level on specific checklist items used by SPs for recording interviewing, physical-examination, and communication skills. The "strong SEs" were trained to score 80% correct on six of the examination's 12 cases (study cases), and the "weak SEs" were trained to score 40% correct on the same six cases. The strong and weak SEs' checklist scores on the study cases were compared through independent, two-tailed t-tests. When there was less than 85% agreement on specific checklist items in each case between the SE training and the SP recording, videotapes of the cases were reviewed; in such cases an SE's performance was the final score agreed upon after review. RESULTS: Seven SEs took the SP examination and were not detected by the SPs. There was a total of 84 discrepancies between predicted and recorded checklist scores across 659 checklist items in 40 encounters scored by the SPs. After correcting the discrepancies based on videotape review, the estimated actual mean score was 77.3% for the strong SEs and 44.0% for the weak SEs, and was higher for the strong SEs in each study case. The overall fidelity of the SEs to their training was estimated to be 97%, and the overall SP accuracy was estimated to be 91%. The videotape review revealed 47 training-scoring discrepancies, most in the area of communication skills. CONCLUSION: This study suggests that SEs can be trained to specific performance levels and may be an effective internal control for a high-stakes SP examination. They may also provide a mechanism for refining scoring checklists and for exploring the validity of SP examinations.     

The neurologic examination

The essentials of the neurological examination are reviewed from the perspective of the emergency medicine practitioner. Important historical information to be obtained, as well as indications and techniques for individual components of the physical examination, are covered. Appropriate and concise methods for documenting the results of the neurological examination in the patient record are also discussed.     

Reliability of the Part II board certification examination in psychiatry: examination stability

OBJECTIVE: The aim of this study was to examine the reliability (examination stability) of the American Board of Psychiatry and Neurology (ABPN) Part II (oral) examination in psychiatry. METHOD: The authors analyzed the consistency (agreement between grades given by two independent examiners) for a 1-year examination cycle using a weighted kappa statistic and compared different parts of the examination (live patient and videotape), different examination sites, different days, and different times of the day. RESULTS: There was no significant difference in agreement between examiners by different parts of the examination, examination site, day of the week, or time of day. CONCLUSIONS: The stability of the Part II ABPN examination in psychiatry is not influenced significantly by the format or site of administration. Candidate performance is the predominant factor in the determination of passing or failing grades.     

Barium enema examination in acute appendicitis

The barium enema examination is a safe technique for the prompt and accurate diagnosis of acute appendicitis. Properly performed and correctly interpreted, the examination can serve as a valuable tool for the diagnosis or exclusion of acute appendicitis when this is not possible by clinical and laboratory procedures.     

Mammography versus clinical examination of the breasts

Using published data from screening trials, this article compares two-modality (mammography and clinical examination) and single-modality (clinical examination alone) screening by evaluating cancer detection rates, program sensitivities, mode of cancer detection in two-modality screening, nodal status at time of detection, survival 10 years post-diagnosis, and breast cancer mortality 10 years after entry. Consistently, two-modality screening achieved higher cancer detection rates and program sensitivity estimates than either modality alone; mammography alone achieved higher rates than clinical examination alone; interval cancer detection rates between screening examinations were higher following clinical examination alone than mammography alone; single-modality screening with mammography failed to detect breast cancers identified by clinical examination alone; the sensitivity of mammography was lower in younger than older women, while the reverse was true for clinical examination; and mammography identified a higher proportion of node-negative breast cancer than clinical examination. We conclude that combining clinical breast examination with mammography is desirable for women age 40-49 because mammography is less sensitive in younger than older women. Careful training and monitoring are, however, as essential with clinical examiners as with mammographers.     

Utility of fine-needle aspiration cytology and frozen-section examination in the operative management of thyroid nodules

Fine-needle aspiration cytology has a high sensitivity for the diagnosis of solitary thyroid nodules. Certain diagnoses involving follicular histologies often cannot be made with needle biopsy alone. The utility of frozen-section examination of thyroid nodules, with particular regard to those lesions with follicular histologies, is also limited. We examined the correlation of fine-needle aspiration cytology and frozen-section examination in solitary thyroid nodules to determine the contribution of frozen-section examination to the operation. We reviewed the fine-needle aspiration cytology, frozen-section examination, and final pathology of 100 consecutive patients undergoing thyroidectomy for a solitary solid thyroid nodule in an 4-year period. The diagnoses were classified as indeterminant, benign, or malignant. The utility and impact of the diagnosis from fine-needle aspiration or frozen section on the operative procedure performed was analyzed. Fine-needle aspiration cytology as a diagnostic test for thyroid nodules demonstrated an indeterminant rate of 23 per cent, with a diagnostic accuracy of 77 and 92 per cent for benign and malignant disease, respectively. In all patients with inaccurate benign diagnosis on fine-needle aspiration cytology, follicular neoplasm was misinterpreted for follicular adenoma or multinodular goiter. In comparing frozen-section results, the indeterminant, benign, and malignant rates were 7, 96, and 64 per cent, respectively. Of the 23 patients with indeterminant results on fine-needle aspiration cytology, the intraoperative frozen-section diagnosis on 4 patients was deferred to permanent section; 18 received accurate cytological diagnosis; and in 1 patient, carcinoma was missed. Overall, the decision about the extent of surgical thyroid resection was changed in only 2 patients based on the frozen-section results. Preoperative evaluation with fine-needle aspiration cytology can accurately and appropriately define the extent of thyroid surgery in most patients with a diagnosis of malignant neoplasm or benign disease. Intraoperative frozen-section examination may be helpful if fine-needle aspiration cytology results are inderminant and in cases of follicular histology as an adjunct for evaluation of the thyroid nodule, but overall, frozen section does not contribute to the management of the thyroid lesion at the time of surgery.     

Epidemiologic significance of detecting shigellae during examination of a healthy population

The authors present the results of examination of the healthy population for shigellae carrier state. The efficacy of detection of shigellae depending on the season of the year and indications to the examination was compared. Analysis of the data obtained indicated that any examination for shigellae during the extraepidemic period was inexpedient. This conclusion was confirmed by the results of selective examination with the use of bacteriological and serological methods.     

Essential food hygiene certificate examination: examination papers reviewed

One thousand papers for the Essential Food Hygiene examination, chosen consecutively from those being processed in RSH House on 6-7 April 1994, were reviewed. The purpose of the review was to assess the impact of the redesigned course and the new examination papers which were introduced from 1 January 1994. The pass rate for the sample was 96.7% which was somewhat lower than the pass rate of 99.2% for the whole of 1993 when the previous examination papers were in use. Out of 1000 papers 33 were failures. Eighteen candidates had obtained the 70% pass mark yet still failed because one of the two questions on temperature control had been answered incorrectly. The requirement to answer both those questions correctly in order to obtain a certificate was introduced with the new examination papers and marked a major change. The conclusion from this snapshot study was that the majority of Tutors and the candidates from their courses had achieved a high degree of examination success. In particular, Tutors had secured a major objective by ensuring that trainees had a clear understanding of the concepts of temperature control. Only a minority were having difficulty. Of the 33 individuals who failed, 18 came from just four of the 108 courses.     

A comparison of the effectiveness of malignancy detection in body fluid examination by the cytopathology and hematology laboratories

BACKGROUND: Body fluid specimens in many institutions are submitted for cytologic examination as well as for examination in a clinical microscopy or hematology laboratory. The cytology laboratory is generally seen as the standard for detection of malignancy, whereas the clinical microscopy laboratory is often depended on predominantly for cell counting and categorization. METHODS: To analyze the effectiveness of the hematology laboratory at detecting malignant fluids, this study retrospectively analyzed reports on 397 body fluid specimens (cerebrospinal, pericardial, peritoneal, and pleural) that were concurrently submitted over a 12-month period to both the cytopathology laboratory and the hematology laboratory. RESULTS: Thirty-seven (9.3%) of the cases were diagnosed as malignant by at least one of the two examinations. The cytopathology examination reported 27 (73%) of the 37 malignant cases as malignant and 30 (81.1%) as at least atypical (27 malignant and 3 inconclusive), and the hematology examination reported 34 (91.9%) as malignant and 36 (97.3%) as at least atypical. A concordant malignant diagnosis was given by both laboratories in 24 (64.9%) of the 37 cases. CONCLUSIONS: These results show that examination of specimens by the hematology laboratory can provide a highly sensitive diagnostic evaluation in addition to its more customary role of providing timely cell counts.     

A comparison of sonographic examinations for trauma performed by surgeons and radiologists

BACKGROUND: It has been demonstrated that surgeons and surgery residents, trained in the focused abdominal sonographic examination, are able to accurately and reliably evaluate trauma patients. Despite this, radiologists have objected to surgeon-performed sonography for several reasons. We set out to compare the accuracy of sonographic examinations performed by surgery residents and radiologists. METHODS: A retrospective review of medical records of all trauma patients who received focused ultrasound examinations from January 1, 1995, through June 30, 1996, at one of two American College of Surgeons-verified Level I trauma centers in the same city was undertaken. Ultrasound examinations were performed by surgery residents at trauma center A (TCA) and by radiologists or radiology residents at trauma center B (TCB). Findings for each patient were compared with the results of computed tomography, diagnostic peritoneal lavage, operative exploration, or observation. Sensitivity, specificity, accuracy, positive predictive value, and negative predictive value were calculated for each group of patients. Comparison of patient charges for the trauma ultrasound examinations at each of the trauma centers was also made. RESULTS: Patient populations at the two centers were similar except that the mean Injury Severity Score at TCB was higher than at TCA (11.74 vs. 9.6). Sensitivity, specificity, accuracy, or negative predictive value were not significantly different between the two cohorts. A significantly lower positive predictive value for examinations performed by surgery residents was noted and attributed to a lower threshold of the surgery residents to confirm their findings by computed tomography. Billing data revealed that the average charge for trauma sonography by radiologists (TCB) was $406.30. At TCA, trauma sonography did not generate a specific charge; however, a $20.00 sum was added to the trauma activation fee to cover ultrasound machine maintenance and supplies. CONCLUSION: Focused ultrasound examination in the trauma suite can be as safely and accurately performed by surgery residents as by radiologists and radiology residents and should be a routine part of the initial trauma evaluation process.     

Macroscopic examination of the placenta immediately following birth

Macroscopic examination of the placenta can be performed readily by the healthcare providers present at the time of birth. It can be accomplished with a basic knowledge of placental pathology and a general understanding of the abnormalities and variations that affect the placenta. These observations can easily be entered into the medical record and may often provide information about intrauterine life that will help the future care of both mother and infant.     

Standardized ultrasound examination for evaluation of instability of the acromioclavicular joint

Anteroposterior X-ray views of both acromioclavicular (AC) joints with 10-kg weights held in each hand are the generally accepted procedure for diagnosis of Tossy I-III grades of AC joint separation. An analogous diagnosis can be made by standardized ultrasound examination. Ten individuals with Tossy-I, 11 with Tossy-II and 8 with Tossy-III instability were examined both radiographically and by B-mode ultrasound. The degree of AC joint separation was uniformly determined on the basis of a calculated index (AC Index = AC joint width of uninjured side/AC joint width of injured side). The mean AC Index for Tossy-I instability determined by ultrasound was 1.0; mean indices of 0.49 and 0.5 were determined for Tossy-II injury by ultrasound and X-ray, respectively, and of 0.21 and 0.2, respectively, for Tossy-III instability. Statistical analysis showed significant differences between the mean AC indices of all three groups (P < 0.0001). We conclude that the reliability of ultrasound examination of AC joint instability is equal to that of radiographic measurement. Standard X-rays of the shoulder remain mandatory only to exclude fracture. The indication for operative stabilization of the AC joint can be established on the basis of the grade of AC joint instability measured by the side-effect-free and cost-effective method of ultrasound examination (AC Index < 0.3 equivalent to Tossy-III instability).     

Pulmonary metastasizing of prostatic carcinoma--source of errors in the ambulatory x-ray examination of lungs

It is reported on eight patients with a primary carcinoma of the prostatic gland who were admitted to the hospital for clinical clarification of conspicuous pulmonary findings. The radiological findings of a general radiograph of the thorax are multiform and for themselves alone they do not permit an etiological coordination. For this reason the demand can be derived in patients of the adequate age group with tumour symptomatology can also be thought of a carcinoma of the prostatic gland and already during the examination in the outpatient department to carry out a palpatory examination and, if possible, to make a general radiograph of spinal column and pelvis, since in these cases the diagnosis can be made already before admission.     

Standardized CT examination of the multitraumatized patient

The aim of this study was to develop a standardized non-helical-CT protocol including head, body and proximal extremities in order to achieve a good time efficiency and diagnostic accuracy in the initial radiological evaluation of the multitraumatized patient. A total of 111 circulatory stable blunt trauma patients, brought in to a trauma level II-III hospital, were examined according to a standardized CT protocol. After examining the head with contiguous 10-mm slices without i.v. contrast medium injection, the trunk was examined with 10-mm slices every 30 mm through thorax-abdomen-pelvis with i.v. contrast medium enhancement (occasionally modified). All data in the medical reports were collected and used as "end-point", and the outcome of the CT examination was compared with this final diagnosis. Mean examination time was 20 min (range 12-32 min). In total, 55 head injuries, 89 thoracic injuries, 27 abdominal/pelvic injuries and 62 fractures were found. Computed tomography correctly identified the injuries, except one brain stem injury, one contusion/rupture of the heart, one hepatic injury, two intestinal injuries, eight vertebral injuries and one joint dislocation. A standardized non-helical-CT examination of the head and body may be achieved in 20 min. Its diagnostic accuracy was high, except for vertebral column injuries, which is why we recommend it as the method of choice for initial radiological examination of multitraumatized patients. When available, helical scanning would improve both examination speed and accuracy.     

Identifying the patient with sleep apnea: upper airway assessment and physical examination

Since the final common pathway for obstructive sleep apnea is obstruction of the upper airway during nocturnal respiration, examination and assessment of the anatomy of the upper airway plays a central role in patient evaluation. Since the upper airway begins at the nose and lips and ends at the larynx, a complete assessment of the upper airway evaluates this entire length of this anatomic region including the bony framework and soft tissue. Though office assessment of these structures does not necessarily mimic the appearance of behavior of these structures during physiologic sleep, the office examination can give important information as to the site of obstruction during sleep that can help direct therapy.     

The adequacy of medical school education in musculoskeletal medicine

A basic familiarity with musculoskeletal disorders is essential for all medical school graduates. The purpose of the current study was to test a group of recent medical school graduates on basic topics in musculoskeletal medicine in order to assess the adequacy of their preparation in this area. A basic-competency examination in musculoskeletal medicine was developed and validated. The examination was sent to all 157 chairpersons of orthopaedic residency programs in the United States, who were asked to rate each question for importance and to suggest a passing score. To assess the criterion validity, the examination was administered to eight chief residents in orthopaedic surgery. The study population comprised all eighty-five residents who were in their first postgraduate year at our institution; the examination was administered on their first day of residency. One hundred and twenty-four (81 per cent) of the 154 orthopaedic residency-program chairpersons who received the survey responded to it. The chairpersons rated twenty-four of the twenty-five questions as at least important. The mean passing score (and standard deviation) that they recommended for the assessment of basic competency was 73.1 +/- 6.8 per cent. The mean score for the eight orthopaedic chief residents was 98.5 +/- 1.07 per cent, and that for the eighty-five residents in their first postgraduate year was 59.6 +/- 12 per cent. Seventy (82 per cent) of the eighty-five residents failed to demonstrate basic competency on the examination according to the chairpersons' criterion. The residents who had taken an elective course in orthopaedic surgery in medical school scored higher on the examination (mean score, 68.4 per cent) than did those who had taken only a required course in orthopaedic surgery (mean score, 57.9 per cent) and those who had taken no rotation in orthopaedic surgery (mean score, 55.9 per cent) (p = 0.005 and p = 0.001, respectively). In summary, seventy (82 per cent) of eighty-five medical school graduates failed a valid musculoskeletal competency examination. We therefore believe that medical school preparation in musculoskeletal medicine is inadequate.     

Correlations between clinical examination/MRI/arthroscopy in the acute traumatic knee

Authors study retrospectively 81 cases of isolated recent (less than three months) knee traumas. These include clinical aspects, NMR, arthroscopy. Clinical examinations have been graded: 1. possible lesion; 2. likely lesions; 3. confirmed lesions. Clinical examinations and NMR results are compared to arthroscopy considered as reference. Clinical examination of acute traumatic knee is essential. Nevertheless, its value for detecting precise lesions is poor, except for knee locking well correlated with meniscal tears. Results show that NMR provide better results than clinical examination. NMR is reliable for detecting tears of posterior cruciate ligament, tears of posterior horn of menisci (sensibility: 93%; specificity: 80%). Its results are less effective for appreciation of lesion of anterior cruciate ligament (sensibility: 88%; specificity: 78%) because of partial tears and functional but not morphologic damage. It is the only method able to evidence osteochondral injuries and soft-tissues associated lesions in traumatic knees. Emergency NMR scans show results no differences in results compared with routine examinations. However, one should keep in mind that negative NMR cannot exclude small cartilaginous lesions and partial tears of anterior cruciate ligament. According to these results and the known qualities of NMR (non invasive), we propose that this type of investigation should be more largely included in diagnostic attitude for acute injured knee. Emergency diagnostic arthroscopy could be efficiently replaced by NMR knee examination.