| Abstract: | A framework for evaluating the performance of a document analysis system is presented. This framework takes into account the task definition for the document analysis system, a data base on which that system is evaluated, the metrics used to evaluate performance, and the generalization of the results achieved beyond the confines of the test. Several recent significant efforts in evaluating document analysis systems are surveyed. How these efforts fit the general framework is discussed. The specific task that was evaluated, the data base used for the evaluation, and the generalization of the derived performance is presented. Most of these projects were designed for limited applications in which the translation of images of text into ASCII was the primary consideration. However, this is only part of what a document analysis system must often calculate. Other, less easily measured tasks, such as the subdivision of a document image into zones that represent regions of graphics, photographs, and text, must also be performed. Generally accepted solutions for measuring the performance of such tasks often do not exist. Several of them are mentioned as areas for future research. © 1996 John Wiley & Sons, Inc. |
