Extracting Rules from Composite Neural Networks for Medical Diagnostic Problems

Recently, neural networks have been applied to many medical diagnostic problems because of their appealing properties, robustness, capability of generalization and fault tolerance. Although the predictive accuracy of neural networks may be higher than that of traditional methods (e.g., statistical methods) or human experts, the lack of explanation from a trained neural network leads to the difficulty that users would hesitate to take the advise of a black box on faith alone. This paper presents a class of composite neural networks which are trained in such a way that the values of the network parameters can be utilized to generate If-Then rules on the basis of preselected meaningful coordinates. The concepts and methods presented in the paper are illustrated through one practical example from medical diagnosis.     

Extraction of Logical Rules from Neural Networks

Three neural-based methods for extraction of logical rules from data are presented. These methods facilitate conversion of graded response neural networks into networks performing logical functions. MLP2LN method tries to convert a standard MLP into a network performing logical operations (LN). C-MLP2LN is a constructive algorithm creating such MLP networks. Logical interpretation is assured by adding constraints to the cost function, forcing the weights to ±1 or 0. Skeletal networks emerge ensuring that a minimal number of logical rules are found. In both methods rules covering many training examples are generated before more specific rules covering exceptions. The third method, FSM2LN, is based on the probability density estimation. Several examples of performance of these methods are presented.     

一种基于神经网络规则提取的新方法

提出了一种基于神经网络对信息系统进行规则提取的新方法。首先用粗糙集对信息系统进行属性约简,然后把条件属性作为输入,决策属性作为最后输出对多层神经网络进行训练。由相关定理对神经网络的运行结果做了理论分析,并以分析结果作为规则提取的重要依据。实验结果验证了新算法的有效性。新算法与几种传统算法相比规则提取的准确率有很大的提高。     

Using Knowledge-Based Neural Networks to Improve Algorithms: Refining the Chou–Fasman Algorithm for Protein Folding

This article describes a connectionist method for refining algorithms represented as generalized finite-state automata. The method translates the rule-like knowledge in an automaton into a corresponding artificial neural network, and then refines the reformulated automaton by applying backpropagation to a set of examples. This technique for translating an automaton into a network extends the KBANN algorithm, a system that translates a set of prepositional rules into a corresponding neural network. The extended system, FSKBANN, allows one to refine the large class of algorithms that can be represented as state-based processes. As a test, FSKBANN is used to improve the Chou–Fasman algorithm, a method for predicting how globular proteins fold. Empirical evidence shows that the multistrategy approach of FSKBANN leads to a statistically-significantly, more accurate solution than both the original Chou–Fasman algorithm and a neural network trained using the standard approach. Extensive statistics report the types of errors made by the Chou–Fasman algorithm, the standard neural network, and the FSKBANN network.     

Discovering Trends in Large Datasets Using Neural Networks

A novel knowledge discovery technique using neural networks is presented. A neural network is trained to learn the correlations and relationships that exist in a dataset. The neural network is then pruned and modified to generalize the correlations and relationships. Finally, the neural network is used as a tool to discover all existing hidden trends in four different types of crimes (murder, rape, robbery, and auto theft) in US cities as well as to predict trends based on existing knowledge inherent in the network.     

Representing Probabilistic Rules with Networks of Gaussian Basis Functions

There is great interest in understanding the intrinsic knowledge neural networks have acquired during training. Most work in this direction is focussed on the multi-layer perceptron architecture. The topic of this paper is networks of Gaussian basis functions which are used extensively as learning systems in neural computation. We show that networks of Gaussian basis functions can be generated from simple probabilistic rules. Also, if appropriate learning rules are used, probabilistic rules can be extracted from trained networks. We present methods for the reduction of network complexity with the goal of obtaining concise and meaningful rules. We show how prior knowledge can be refined or supplemented using data by employing either a Bayesian approach, by a weighted combination of knowledge bases, or by generating artificial training data representing the prior knowledge. We validate our approach using a standard statistical data set.     

Knowledge Extraction from Transducer Neural Networks

Previously neural networks have shown interesting performance results for tasks such as classification, but they still suffer from an insufficient focus on the structure of the knowledge represented therein. In this paper, we analyze various knowledge extraction techniques in detail and we develop new transducer extraction techniques for the interpretation of recurrent neural network learning. First, we provide an overview of different possibilities to express structured knowledge using neural networks. Then, we analyze a type of recurrent network rigorously, applying a broad range of different techniques. We argue that analysis techniques, such as weight analysis using Hinton diagrams, hierarchical cluster analysis, and principal component analysis may be useful for providing certain views on the underlying knowledge. However, we demonstrate that these techniques are too static and too low-level for interpreting recurrent network classifications. The contribution of this paper is a particularly broad analysis of knowledge extraction techniques. Furthermore, we propose dynamic learning analysis and transducer extraction as two new dynamic interpretation techniques. Dynamic learning analysis provides a better understanding of how the network learns, while transducer extraction provides a better understanding of what the network represents.     

神经网络规则提取及其在特征带识别中的应用

基于神经网络的规则提取通常只是针对某些特定的数据集,缺乏领域专家的鉴别,实际应用效果难以判定。本文针对分子标记特征带识别这一实际问题,在准确率损失尽量小的情况下剪枝训练好的神经网络,利用同符号规则提取算法（SSRE）抽取规则,避免了繁琐的规则抽取过程。实验表明此方法的效果良好,结果得到生物专家的认可。     

FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks

Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network retraining. Given a fully connected trained feedforward network with a single hidden layer, FERNN first identifies the relevant hidden units by computing their information gains. For each relevant hidden unit, its activation values is divided into two subintervals such that the information gain is maximized. FERNN finds the set of relevant network connections from the input units to this hidden unit by checking the magnitudes of their weights. The connections with large weights are identified as relevant. Finally, FERNN generates rules that distinguish the two subintervals of the hidden activation values in terms of the network inputs. Experimental results show that the size and the predictive accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.     

基于聚类遗传算法的神经网络规则抽取及应用

提出了一种基于Gabor滤波器和神经网络规则抽取的烘焙面包品质分类方法。滤波器对烘焙面包切片区域灰度图像直接进行小波变换,用能量均值"和均方差!来表示灰度图像的纹理特征,并基于对隐层神经元输出值聚类的遗传算法实现了对面包品质分类的规则抽取。实验结果表明了该方法的实用性和可行性。