Integrating computer vision with web‐based knowledge for medical diagnostic assistance

Abstract: The analysis of medical documents necessitates context recognition for diverse purposes such as classification, performance analysis and decision making. Traditional methods of context recognition have focused on the textual part of documents. Images, however, provide a rich source of information that can support the context recognition process. A method is proposed for integrating computer vision in context recognition using the web as a knowledge base. The method is implemented on medical case studies to determine the main symptoms or achieve possible diagnoses. In experiments the method for integrating computer vision in context recognition achieves better results than term frequency and inverse document frequency and only context recognition. The proposed method can serve as a basis for an image and text based decision support system to assist the physician in reviewing medical records.