Extracting rules from a (fuzzy/crisp) recurrent neural network using a self‐organizing map

Although the extraction of symbolic knowledge from trained feedforward neural networks has been widely studied, research in recurrent neural networks (RNN) has been more neglected, even though it performs better in areas such as control, speech recognition, time series prediction, etc. Nowadays, a subject of particular interest is (crisp/fuzzy) grammatical inference, in which the application of these neural networks has proven to be suitable. In this paper, we present a method using a self‐organizing map (SOM) for extracting knowledge from a recurrent neural network able to infer a (crisp/fuzzy) regular language. Identification of this language is done only from a (crisp/fuzzy) example set of the language. © 2000 John Wiley & Sons, Inc.     

Survey and critique of techniques for extracting rules from trained artificial neural networks

It is becoming increasingly apparent that, without some form of explanation capability, the full potential of trained artificial neural networks (ANNs) may not be realised. This survey gives an overview of techniques developed to redress this situation. Specifically, the survey focuses on mechanisms, procedures, and algorithms designed to insert knowledge into ANNs (knowledge initialisation), extract rules from trained ANNs (rule extraction), and utilise ANNs to refine existing rule bases (rule refinement). The survey also introduces a new taxonomy for classifying the various techniques, discusses their modus operandi, and delineates criteria for evaluating their efficacy.     

Symbolic interpretation of artificial neural networks

Hybrid intelligent systems that combine knowledge-based and artificial neural network systems typically have four phases, involving domain knowledge representation, mapping of this knowledge into an initial connectionist architecture, network training and rule extraction, respectively. The final phase is important because it can provide a trained connectionist architecture with explanation power and validate its output decisions. Moreover, it can be used to refine and maintain the initial knowledge acquired from domain experts. In this paper, we present three rule extraction techniques. The first technique extracts a set of binary rules from any type of neural network. The other two techniques are specific to feedforward networks, with a single hidden layer of sigmoidal units. Technique 2 extracts partial rules that represent the most important embedded knowledge with an adjustable level of detail, while the third technique provides a more comprehensive and universal approach. A rule-evaluation technique, which orders extracted rules based on three performance measures, is then proposed. The three techniques area applied to the iris and breast cancer data sets. The extracted rules are evaluated qualitatively and quantitatively, and are compared with those obtained by other approaches     

Providing understanding of the behavior of feedforward neuralnetworks

The advent of artificial neural networks has stirred the imagination of many in the field of knowledge acquisition. There is an expectation that neural networks will play an important role in automating knowledge acquisition and encoding, however, the problem solving knowledge of a neural network is represented at a subsymbolic level and hence is very difficult for a human user to comprehend. One way to provide an understanding of the behavior of neural networks is to extract their problem solving knowledge in terms of rules that can be provided to users. Several papers which propose extracting rules from feedforward neural networks can be found in the literature, however, these approaches can only deal with networks with binary inputs. Furthermore, certain approaches lack theoretical support and their usefulness and effectiveness are debatable. Upon carefully analyzing these approaches, we propose a method to extract fuzzy rules from networks with continuous-valued inputs. The method was tested using a real-life problem (decision-making by pilots involving combat situations) and found to be effective.     

A machine learning method for extracting symbolic knowledge from recurrent neural networks

Neural networks do not readily provide an explanation of the knowledge stored in their weights as part of their information processing. Until recently, neural networks were considered to be black boxes, with the knowledge stored in their weights not readily accessible. Since then, research has resulted in a number of algorithms for extracting knowledge in symbolic form from trained neural networks. This article addresses the extraction of knowledge in symbolic form from recurrent neural networks trained to behave like deterministic finite-state automata (DFAs). To date, methods used to extract knowledge from such networks have relied on the hypothesis that networks' states tend to cluster and that clusters of network states correspond to DFA states. The computational complexity of such a cluster analysis has led to heuristics that either limit the number of clusters that may form during training or limit the exploration of the space of hidden recurrent state neurons. These limitations, while necessary, may lead to decreased fidelity, in which the extracted knowledge may not model the true behavior of a trained network, perhaps not even for the training set. The method proposed here uses a polynomial time, symbolic learning algorithm to infer DFAs solely from the observation of a trained network's input-output behavior. Thus, this method has the potential to increase the fidelity of the extracted knowledge.     

Rule revision with recurrent neural networks

Recurrent neural networks readily process, recognize and generate temporal sequences. By encoding grammatical strings as temporal sequences, recurrent neural networks can be trained to behave like deterministic sequential finite-state automata. Algorithms have been developed for extracting grammatical rules from trained networks. Using a simple method for inserting prior knowledge (or rules) into recurrent neural networks, we show that recurrent neural networks are able to perform rule revision. Rule revision is performed by comparing the inserted rules with the rules in the finite-state automata extracted from trained networks. The results from training a recurrent neural network to recognize a known non-trivial, randomly-generated regular grammar show that not only do the networks preserve correct rules but that they are able to correct through training inserted rules which were initially incorrect (i.e. the rules were not the ones in the randomly generated grammar)     

Extracting Refined Rules from Knowledge-Based Neural Networks

Neural networks, despite their empirically proven abilities, have been little used for the refinement of existing knowledge because this task requires a three-step process. First, knowledge must be inserted into a neural network. Second, the network must be refined. Third, the refined knowledge must be extracted from the network. We have previously described a method for the first step of this process. Standard neural learning techniques can accomplish the second step. In this article, we propose and empirically evaluate a method for the final, and possibly most difficult, step. Our method efficiently extracts symbolic rules from trained neural networks. The four major results of empirical tests of this method are that the extracted rules 1) closely reproduce the accuracy of the network from which they are extracted; 2) are superior to the rules produced by methods that directly refine symbolic rules; 3) are superior to those produced by previous techniques for extracting rules from trained neural networks; and 4) are human comprehensible. Thus, this method demonstrates that neural networks can be used to effectively refine symbolic knowledge. Moreover, the rule-extraction technique developed herein contributes to the understanding of how symbolic and connectionist approaches to artificial intelligence can be profitably integrated.     

Inverting feedforward neural networks using linear and nonlinearprogramming

The problem of inverting trained feedforward neural networks is to find the inputs which yield a given output. In general, this problem is an ill-posed problem. We present a method for dealing with the inverse problem by using mathematical programming techniques. The principal idea behind the method is to formulate the inverse problem as a nonlinear programming problem, a separable programming (SP) problem, or a linear programming problem according to the architectures of networks to be inverted or the types of network inversions to be computed. An important advantage of the method over the existing iterative inversion algorithm is that various designated network inversions of multilayer perceptrons and radial basis function neural networks can be obtained by solving the corresponding SP problems, which can be solved by a modified simplex method. We present several examples to demonstrate the proposed method and applications of network inversions to examine and improve the generalization performance of trained networks. The results show the effectiveness of the proposed method.     

Near field acoustic localization under unfavorable conditions using feedforward neural network for processing time difference of arrival

Using time difference of arrival (TDOA) is one of the two approaches that utilize time delay for acoustic source localization. Combining the obtained TDOAs together with geometrical relationships within acoustic components results in a system of hyperbolic equations. Solving these hyperbolic equations is not a trivial procedure especially in the case of a large number of microphones. The solution is additionally compounded by uncertainties of different backgrounds. The paper investigates the performance of neural networks in modelling a hyperbolic positioning problem using a feedforward neural network as a representative. For experimental purposes, more than 2000 sound files were recorded by 8 spatially disposed microphones, for as many arbitrarily chosen acoustic source positions. The samples were corrupted by high level correlated noise and reverberation. Using cross-correlation, with previous signal pre-processing, TDOAs were evaluated for every pair of microphones. On the basis of the obtained TDOAs and accurate sound source positions, the neural network was trained to perform sound source localization. The performance was examined using a large number of samples in terms of different acoustic sensors setups, network configurations and training parameters. The experiment provided useful guidelines for the practical implementation of feedforward neural networks in the near-field acoustic localization. The procedure does not require substantial knowledge of signal processing and that is why it is suitable for a broad range of users.     

On the construction and training of reformulated radial basis function neural networks

Presents a systematic approach for constructing reformulated radial basis function (RBF) neural networks, which was developed to facilitate their training by supervised learning algorithms based on gradient descent. This approach reduces the construction of radial basis function models to the selection of admissible generator functions. The selection of generator functions relies on the concept of the blind spot, which is introduced in the paper. The paper also introduces a new family of reformulated radial basis function neural networks, which are referred to as cosine radial basis functions. Cosine radial basis functions are constructed by linear generator functions of a special form and their use as similarity measures in radial basis function models is justified by their geometric interpretation. A set of experiments on a variety of datasets indicate that cosine radial basis functions outperform considerably conventional radial basis function neural networks with Gaussian radial basis functions. Cosine radial basis functions are also strong competitors to existing reformulated radial basis function models trained by gradient descent and feedforward neural networks with sigmoid hidden units.     

神经网络规则抽取

神经网络是一种黑箱模型，其学习到的知识蕴涵在大量连接权中，不仅影响了用户对利用神经计算技术构建智能系统的信心，还阻碍了神经网络技术在数据挖掘领域的应用，由于对神经网络规则抽取进行研究有助于解决上述问题，因此该领域已成为机器学习和神经计算界的研究热点，介绍了神经网络规则抽取研究的历史，综述了国际研究现状，对关于这方面研究的不同看法进行了讨论，并指出该领域中一些值得进一步研究的内容。     

Using an artificial neural network to detect activations during ventricular fibrillation.

Ventricular fibrillation is a cardiac arrhythmia that can result in sudden death. Understanding and treatment of this disorder would be improved if patterns of electrical activation could be accurately identified and studied during fibrillation. A feedforward artificial neural network using backpropagation was trained with the Rule-Based Method and the Current Source Density Method to identify cardiac tissue activation during fibrillation. Another feedforward artificial neural network that used backpropagation was trained with data preprocessed by those methods and the Transmembrane Current Method. Staged training, a new method that uses different sets of training examples in different stages, was used to improve the ability of the artificial neural networks to detect activation. Both artificial neural networks were able to correctly classify more than 92% of new test examples. The performance of both artificial neural networks improved when staged training was used. Thus, artificial neural networks may beuseful for identifying activation during ventricular fibrillation.     

An evaluation of quantum neural networks in the detection of epileptic seizures in the neonatal electroencephalogram

This paper presents the results of an experimental study that evaluated the ability of quantum neural networks (QNNs) to capture and quantify uncertainty in data and compared their performance with that of conventional feedforward neural networks (FFNNs). In this work, QNNs and FFNNs were trained to classify short segments of epileptic seizures in neonatal EEG. The experiments revealed significant differences between the internal representations created by trained QNNs and FFNNs from sample information provided by the training data. The results of this experimental study also confirmed that the responses of trained QNNs are more reliable indicators of uncertainty in the input data compared with the responses of trained FFNNs.     

Genetics-based machine learning for the assessment of certainneuromuscular disorders

Clinical electromyography (EMG) provides useful information for the diagnosis of neuromuscular disorders. The utility of artificial neural networks (ANN's) in classifying EMG data trained with backpropagation or Rohonen's self-organizing feature maps algorithm has recently been demonstrated. The objective of this study is to investigate how genetics-based machine learning (GBML) can be applied for diagnosing certain neuromuscular disorders based on EMG data. The effect of GBML control parameters on diagnostic performance is also examined. A hybrid diagnostic system is introduced that combines both neural network and GBML models. Such a hybrid system provides the end-user with a robust and reliable system, as its diagnostic performance relies on more than one learning principle. GBML models demonstrated similar performance to neural-network models, but with less computation. The diagnostic performance of neural network and GBML models is enhanced by the hybrid system.     

神经网络在文本分类上的一种应用

现有的文本分类方法在知识获取方面存在不足。该文针对某种应用需求,提出了人工神经网络和文本分类结合的一种文本分类方法。采用特征词的向量空间来描述文本,利用人工神经网络的良好的学习能力,通过对文本样本集进行训练,从中提取出对文本分类的知识,再利用神经网络和所获得的分类知识实现对文本的分类。     

FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks

Before symbolic rules are extracted from a trained neural network, the network is usually pruned so as to obtain more concise rules. Typical pruning algorithms require retraining the network which incurs additional cost. This paper presents FERNN, a fast method for extracting rules from trained neural networks without network retraining. Given a fully connected trained feedforward network with a single hidden layer, FERNN first identifies the relevant hidden units by computing their information gains. For each relevant hidden unit, its activation values is divided into two subintervals such that the information gain is maximized. FERNN finds the set of relevant network connections from the input units to this hidden unit by checking the magnitudes of their weights. The connections with large weights are identified as relevant. Finally, FERNN generates rules that distinguish the two subintervals of the hidden activation values in terms of the network inputs. Experimental results show that the size and the predictive accuracy of the tree generated are comparable to those extracted by another method which prunes and retrains the network.     

Multiple network fusion using fuzzy logic

Multiplayer feedforward networks trained by minimizing the mean squared error and by using a one of c teaching function yield network outputs that estimate posterior class probabilities. This provides a sound basis for combining the results from multiple networks to get more accurate classification. This paper presents a method for combining multiple networks based on fuzzy logic, especially the fuzzy integral. This method non-linearly combines objective evidence, in the form of a network output, with subjective evaluation of the importance of the individual neural networks. The experimental results with the recognition problem of on-line handwriting characters show that the performance of individual networks could be improved significantly.     

ANN-DT: an algorithm for extraction of decision trees fromartificial neural networks

Although artificial neural networks can represent a variety of complex systems with a high degree of accuracy, these connectionist models are difficult to interpret. This significantly limits the applicability of neural networks in practice, especially where a premium is placed on the comprehensibility or reliability of systems. A novel artificial neural-network decision tree algorithm (ANN-DT) is therefore proposed, which extracts binary decision trees from a trained neural network. The ANN-DT algorithm uses the neural network to generate outputs for samples interpolated from the training data set. In contrast to existing techniques, ANN-DT can extract rules from feedforward neural networks with continuous outputs. These rules are extracted from the neural network without making assumptions about the internal structure of the neural network or the features of the data. A novel attribute selection criterion based on a significance analysis of the variables on the neural-network output is examined. It is shown to have significant benefits in certain cases when compared with the standard criteria of minimum weighted variance over the branches. In three case studies the ANN-DT algorithm compared favorably with CART, a standard decision tree algorithm.     

Knowledge distilling based model compression and feature learning in fault diagnosis

Recently, there has been interest in developing diagnosis methods that combine model-based and data-driven diagnosis. In both approaches, selecting the relevant measurements or extracting important features from historical data is a key determiner of the success of the algorithm. Recently, deep learning methods have been effective in automating the feature selection process. Autoencoders have been shown to be an effective neural network configuration for extracting features from complex data, however, they may also learn irrelevant features. In addition, end-to-end classification neural networks have also been used for diagnosis, but like autoencoders, this method may also learn unimportant features thus making the diagnostic inference scheme inefficient. To rapidly extract significant fault features, this paper employs end-to-end networks and develops a new feature extraction method based on importance analysis and knowledge distilling. First, a set of cumbersome neural network models are trained to predict faults and some of their internal values are defined as features. Then an occlusion-based importance analysis method is developed to select the most relevant input variables and learned features. Finally, a simple student neural network model is designed based on the previous analysis results and an improved knowledge distilling method is proposed to train the student model. Because of the way the cumbersome networks are trained, only fault features are learned, with the importance analysis further pruning the relevant feature set. These features can be rapidly generated by the student model. We discuss the algorithms, and then apply our method to two typical dynamic systems, a communication system and a 10-tank system employed to demonstrate the proposed approach.     

Recurrent neural networks and robust time series prediction

We propose a robust learning algorithm and apply it to recurrent neural networks. This algorithm is based on filtering outliers from the data and then estimating parameters from the filtered data. The filtering removes outliers from both the target function and the inputs of the neural network. The filtering is soft in that some outliers are neither completely rejected nor accepted. To show the need for robust recurrent networks, we compare the predictive ability of least squares estimated recurrent networks on synthetic data and on the Puget Power Electric Demand time series. These investigations result in a class of recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component. Conventional least squares methods of fitting NARMA(p,q) neural network models are shown to suffer a lack of robustness towards outliers. This sensitivity to outliers is demonstrated on both the synthetic and real data sets. Filtering the Puget Power Electric Demand time series is shown to automatically remove the outliers due to holidays. Neural networks trained on filtered data are then shown to give better predictions than neural networks trained on unfiltered time series.