Comparison of oral examination and electronic examination using paired multiple-choice questions

The aim of the present research was to compare the use of multiple-choice questions (MCQs) as an examination method against the oral examination (OE) method. MCQs are widely used and their importance seems likely to grow, due to their inherent suitability for electronic assessment. However, MCQs are influenced by the tendency of examinees to guess answers, warranting research concerning scoring rules different from the simple positive-grades-only scores rule. Alternatively, OE is used in tertiary education, since it enables the assessment of intellectual capabilities and personal traits to a level not found in most other examination formats. However, the significant resource requirements of OE, especially in structured forms, might excessively strain the resources of academic institutions. In the present study, an MCQ test was given to examinees, in the framework of a computer-based learning system. The same examinees took also an OE possessing elements of structure, with three examiners concurrently and independently grading each of the examinees. In the MCQs examination a set of pairs of MCQs was composed. The MCQs in each pair were similar concerning the same topic, but this similarity was not evident for an examinee that did not possess adequate knowledge on the topic addressed in the questions of the pair. The scoring of the paired questions avoided the procedure of mixed-scoring, i.e., both positive and negative markings, while at the same time a pair-wise bonus/penalty scoring rule was adopted. The results of the “paired” MCQs examination, when using the pair-wise scoring rule, were statistically indistinguishable with the grades produced by the OE, when made to the same sample of students, on the same topics and with the same levels of difficult. Both the results of the paired MCQs examination, when using the pair-wise scoring rule, and the OE results differed significantly from those obtained by scoring the same MCQs using a positive-grades-only scoring rule that ignored the pairing of MCQs.