Controlling for chance agreement in the validation of medical expert systems with no gold standard: PNEUMON-IA and RENOIR revisited.

In the validation of medical expert systems, agreement among different human specialists on a random sample of cases may be taken as a substitute to a missing gold standard. Distance measures between pairs of experts, extensively described in previous studies, do not take into account the influence of chance-expected agreement. A weighted kappa index, with three different weighting schemes, is proposed as an alternative to be applied in the general situation of N cases assessed by E experts about K possible diagnoses, each of them qualified with one of G ordinal categories. A hierarchical cluster analysis, applied to the kappa matrices generated, allows for the classification of the expert system among clinical specialists, providing a relative assessment of its diagnostic ability. The above methodology is applied to the validation of two medical expert systems, PNEUMON-IA and RENOIR.     

Overcoming deficiencies of the rule-based medical expert system

One of the current deficiencies of the rule-based expert system is its static nature. As these systems are applied to medicine, this shortcoming becomes accentuated by: the rapid speed at which new knowledge is generated, the regional differences associated with the expression of many diseases, and the rate at which patient demographics and disease incidence change over time. This research presents a solution to the static nature of the rule-based expert system by proposing a hybrid system. This system consists of an expert system and a statistical analysis system linked to a patient database. The additional feature of a rule base manager which initiates automatic database analysis to refresh the statistical correlation of each rule ensures a dynamic, current, statistically accurate rule base. The philosophical differences between data and knowledge are also addressed as they apply to this type of hybrid system. The system is then used to generate four rule bases from different knowledge sources. These rule bases are then compared.     

Validation of biochemical laboratory results using the DNSev expert system

In a modern hospital biochemical laboratory, the efficiency and quality of the analysis result production process are fundamental. With respect to quality, an important step in the process is validation. In this step, laboratory physicians, who are physicians specialized in laboratory work, check the analysis result reports in order to verify that no error has occurred during their production. The application described in this paper is an expert system named DNSev. DNSev has been developed in order to improve the quality of the validation process performed by a specific laboratory information system (LIS), called ITALAB C/S, a system used in about thirty percent of Italian hospital laboratories. Objectives achieved by DNSev are: analysis result validation support (medical laboratory expertise in the process is translated into rules and automatically applied by DNSev), help for laboratory automation (checks that are usually manually executed are now automatically executed), clarity (reasoning performed by DNSev in issuing alarms is documented in order to explain it to laboratory physicians), flexibility (new types of reasoning can be easily added to the system by simply upgrading its knowledge base), reliability (checks may be tailored, based on patient characteristics) and time saving and cost reduction. During the development of DNSev, the knowledge acquisition and elicitation task was performed by interviewing laboratory physicians, and also by using available documents and laboratory guidelines. In order to conduct a significant trial test, we installed DNSev in the centralized biochemical laboratory of ‘Sant'Orsola-Malpighi’ Hospital in Bologna (Italy). This is one of the largest Italian biochemical laboratories, managed entirely by ITALAB C/S LIS. During this test we achieve a time saving of around 63% for each analysis request and a reduction on the overall number of analysis requests to be manually examined by laboratory physicians by around 20–25%. About performance of DNSev checks we achieved good accuracy and sensibility levels and a very low false normal level. These results demonstrate that an expert system may be a valid solution for improving quality and efficiency of well-defined medical tasks.     

A probabilistic interpretation of the medical expert system CADIAG-2

CADIAG-2 is a well-known expert system aimed at providing support for medical diagnose in the field of internal medicine. CADIAG-2 consists of a knowledge base in the form of a set of IF-THEN rules that relate distinct medical entities, in this paper interpreted as conditional probabilistic statements, and an inference engine constructed upon methods of fuzzy set theory. The aim underlying this paper is the understanding of the logical structure of the inference in CADIAG-2. To that purpose, we provide a (probabilistic) logical formalisation of the inference of the system and check its adequacy with probabilistic logic.     

医疗辅助诊断专家系统中的规则可信度计算方法

采用粗糙隶属函数计算规则可信度可减少确定可信度（Certainty Factor,CF）的主观因素,使规则可信度更加精确客观。给出计算孤立性肺结节辅助诊断专家系统中规则可信度的实例,证明该方法可提高应用可信度理论进行不确定性推理的可靠性。     

基于机器学习的计算机辅助诊断模型的研究

对基于自学习功能的计算机辅助医学诊断专家系统的应用进行研究。应用基于模糊数学的方法建立急性脑梗塞早期分型医学诊断模型,利用VB语言进行系统的开发,并应用于临床诊断。对山西医科大学神经科已确诊的典型病倒145份进行诊断,计算机与实际诊断结果具有较好的贴近度。在随后对86例门诊患者的预测诊断中,诊断正确率达到了90.7%。该诊断模型与实际诊断结果具有良好的拟好的效果。     

An expert system for managing medical appropriateness criteria based on computer algebra techniques

This article presents the development of an expert system for managing medical appropriateness criteria together with an outline of its theoretical foundations. Techniques borrowed from computer algebra (Gröbner bases) are applied to this field of medicine.

The steps of the expert system construction process are as follows. First, the knowledge provided in table format by experts in coronary diseases is translated into a set of production rules of a rule-based expert system (RBES). Kleene's three-valued logic augmented with modal operators is chosen in order to manage uncertainty.     

Neural expert system using fuzzy teaching input and its application to medical diagnosis

This paper first proposes a fuzzy neural network and the learning method using fuzzy teaching input. As an application, a fuzzy neural expert system (FNES) for diagnosing hepatobiliary disorders has been developed. We used a real medical database containing the results of nine biochemical tests of four hepatobiliary disorders. After learning by using training data (373 patients), the proposed system correctly diagnosed 77.3% of test (external) data from 163 previously unseen patients and correctly diagnosed 100% of the training data. Conversely, the diagnostic accuracy of the linear discriminant analysis was 63.2% of the test data and 67.0% of the training data.     

Evaluating expert system prototypes

There is a disparity between the multitude of apparently successful expert system prototypes and the scarcity of expert systems in real everyday use. Modern tools make it deceptively easy to make reasonable prototypes, but these prototypes are seldom made subject to serious evaluation. Instead the development team confronts their product with a set of cases, and the primary evaluation criterion is the percentage of correct answers: we are faced with a 95% syndrome. Other aspects related to the use of the system are almost ignored. There is still a long way to go from a promising prototype to a final system.It is maintained in the article that a useful test must be performed by future users in a situation that is as realistic as possible. If this is not done claims of usefulness cannot be justified. It is also stated that prototyping does not make traditional analysis and design obsolete, although the contents of these activities will change.In order to discuss the effects of using the systems a distinction between expert systems as media, tools and experts is proposed.     

MESTRIMAN: An expert system for medical triage and clinical management of patients in catastrophes

Providing appropriate medical assistance in catastrophes is not an easy problem to cope with. Some of the problems involved in the task are the diversity of catastrophes and medical emergencies, the insufficient number of experts usually available during a given catastrophe, the necessity of improving medical performace time, and the lack of coordination of the resources that may be used. In this respect, the application of both communication and computer technologies may provide important and useful support in most of the different phases involved in the management of catastrophes. In this paper we describe “MESTRIMAN,” an expert system designed to provide medical assistance in catastrophes. “MESTRIMAN” is a module of SIAC, a computer system designed to manage relevant information and data about a wide range of different catastrophes and capable of performing as the central coordinator of the available resources.     

A castability expert system

The castability analysis of a part, its cost estimation and preparation of competitive quotations typically requires years of experience, is time consuming and is dependent on the expert personnel available. Considerable amount of personnel time is also involved in preparing these quotations. An expert system is ideally suited for this application as it can automate the castability analysis and the quotation preparation process. This will not only substantially improve productivity and consistency, but also the accuracy of the process. This paper describes the development and implementation of an expert system for a typical medium sized company, which receives approximately thousand requests for quotations every year. A Castability Expert System apart from reducing time and increasing accuracy, will enable people with little experience to analyze the part, estimate cost and prepare quotations without the assistance of an expert, whose time can be better utilized in other areas.     

The Fossil expert system

Abstract: Fossil is an expert system for Palaeontology, written in Prolog. It consists of a knowledge base and three interfaces: a collector's 'query' interface; an expert's 'update' interface and a student's 'review' interface. As such, it covers the major uses of a knowledge base in palaeontology. The underlying model of expert system use and its supporting tools are described. This view is an alternative to that provioded by expert system shells of the Apes type. Such an approach has proved useful in Palaeontology and is likely to be applicable to other areas where the knowledge is arranged taxonomically. The use of Prolog as an implementation tool is briefly discussed.     

Specifying an expert system

The success of numerous expert systems in practical applications warrants a more formal approach to their development and evaluation. Reliability assurance of expert systems requires a methodology for the specification and evaluation of these systems. Expert systems are a new class of software system, but some traditional techniques of software development may be adapted to their construction. However, the specification of an expert system differs from that of a more traditional software program in that parts of the specification are permitted to be only partially described when development starts.

Specifications have two important purposes: as contracts between suppliers and clients, and as blueprints for implementation. A specification consists of a problem specification and a solution specification. The problem specification plays the role of contract and states explicitly what the problem to be solved is, and the constraints that the final product must satisfy. The solution specification plays the role of blueprint and has two major aspects: analyzing how a human expert solves the problem, and proposing an equivalent automated solution. We propose an approach to the specification of expert systems that is flexible, yet rigorous enough to cover the important features of a wide range of potential expert system applications. We describe fully each of the components of an expert system specification and we relate specification to the issues of evaluation and maintenance of expert systems.     

Obligations of the expert system builder: meeting the needs of the user.

Builders of expert systems have generally accepted the principle that computer software should not be subject to government regulation if health care practitioners can be expected to interpret and apply the systems intelligently. The purpose of this paper is to identify the information that builders must make available to permit health care practitioners to exercise their clinical judgment in interpreting and applying the output of computing systems.     

Development of an expert system advisor for anaesthetic control.

A review is given of numerous approaches which have been taken to provide automated control of depth of anaesthesia. Most of these approaches use a single indicator for anaesthetic state, and do not perform adequately for such a complex system. It is desirable, therefore, to merge a number of qualitative clinical signs and quantitative on-line measurements to provide decision-support for control of anaesthetic depth. The paper describes such an expert system called Resac (Real time Expert System for Advice and Control). Details of the knowledge representation and inference structure are given, together with the method adopted for propagating uncertainty measures, combining fuzzy information and merging quantitative and qualitative indicators. The importance of good human machine interface design is shown via the GEM-based graphics of Resac.     

A performance evaluation of the expert system ANEMIA

This paper reports the results of an evaluation study of the current level of performance given by ANEMIA, a knowledge-based consultation system addressing the clinical problem of managing anemic patients. ANEMIA was developed on a mainframe using the AI programming scheme EXPERT and then translated into a version running on a personal computer. At present the system is able to provide assistance in the diagnosis and management of 65 disease entities. After extensive local testing of accuracy, completeness, and consistency of the knowledge base included into ANEMIA, we designed a study to evaluate whether the system is able to appropriately mirror also the reasoning of well-known hematologists other than those who provided the knowledge. We were also interested in testing whether there were conflicting opinions among hematologists. Thus, we designed a validation study in which ANEMIA's performance could be compared with that of six hematologists and the interexpert consensus evaluated. ANEMIA's overall performance was judged acceptable in 87% (26/30) of the cases, while expert evaluators agreed with their colleagues in 90% (27/30) of them. A low interexpert consensus was found: considering the ratings given by different hematologists to the same ANEMIA performance, complete agreement occurred only 47% of the time.     

Hypermedia and randomized algorithms for medical expert systems

KNET is an environment for constructing probabilistic, knowledge-intensive systems within the axiomatic framework of decision theory. The KNET architecture defines a complete separation between the hypermedia user interface on the one hand, and the representation and management of expert opinion on the other. KNET offers a choice of algorithms for probabilistic inference. We and our coworkers have used KNET to build consultation systems for lymph-node pathology, bone-marrow transplantation therapy, clinical epidemiology, and alarm management in the intensive-care unit. Most important, KNET contains a randomized approximation scheme (RAS) for the difficult and almost certainly intractable problem of Bayesian inference. Our algorithm can, in many circumstances, perform efficient approximate inference in large and richly interconnected models of medical diagnosis. In this article, we describe the architecture of KNET, construct a randomized algorithm for probabilistic inference, and analyze the algorithm's performance. Finally, we characterize our algorithms' empiric behavior and explore its potential for parallel speedups. From design to implementation, then, KNET demonstrates the crucial interaction between theoretical computer science and medical informatics.