Extracting decision trees from trained neural networks

In this paper we present a methodology for extracting decision trees from input data generated from trained neural networks instead of doing it directly from the data. A genetic algorithm is used to query the trained network and extract prototypes. A prototype selection mechanism is then used to select a subset of the prototypes. Finally, a standard induction method like ID3 or C5.0 is used to extract the decision tree. The extracted decision trees can be used to understand the working of the neural network besides performing classification. This method is able to extract different decision trees of high accuracy and comprehensibility from the trained neural network.     

基于改进型遗传算法的前馈神经网络优化设计

阐明了遗传算法和神经网络结合的可行性，提出了一种改进的面向神经网络权值学习的遗传算法。通过对XOR问题的实验，显示出其快速学习网络权值的能力，且能摆脱局部极值的困扰和初始权值的限制，从各方面都表现出优于标准遗传算法和BP算法的性能。     

一种基于改进DRNN网络的决策树构建方法

决策树是数据挖掘和归纳学习的重要方法。本文介绍了ID3算法,ID3算法存在着倾向于取值较多属性的缺点;神经网络也可以用来分类,但是神经网络不易于分类规则的提取。在遗传算法优化的DRNN网络的基础上,提出了使用差量法构建决策树的方法。该方法既具有神经网络分类的高精度,而且分类规则比较直观。实验数据分析表明,本文提出的方法更加接近实际情况,能够更好的进行预测和分类。     

Online adaptive decision trees

Decision trees and neural networks are widely used tools for pattern classification. Decision trees provide highly localized representation, whereas neural networks provide a distributed but compact representation of the decision space. Decision trees cannot be induced in the online mode, and they are not adaptive to changing environment, whereas neural networks are inherently capable of online learning and adpativity. Here we provide a classification scheme called online adaptive decision trees (OADT), which is a tree-structured network like the decision trees and capable of online learning like neural networks. A new objective measure is derived for supervised learning with OADT. Experimental results validate the effectiveness of the proposed classification scheme. Also, with certain real-life data sets, we find that OADT performs better than two widely used models: the hierarchical mixture of experts and multilayer perceptron.     

Research on delamination monitoring for composite structures based on HHGA–WNN

Due to the deficiencies of the training algorithms for available wavelet neural network used for structural health monitoring, a new hybrid hierarchy genetic algorithm was introduced by combining hierarchy genetic algorithm and least-square method to improve the learning procedure of wavelet neural network. The hybrid algorithm was able to determine the structure and parameters of the wavelet neural network simultaneously. In this algorithm, adaptive crossover and mutation probability were used to accelerate the genetic speed and avoid the occurrence of prematurity. The modal frequencies of a glass/epoxy laminates beam with varying assumed delamination sizes and locations were computed using finite element method and fed into the wavelet neural network to predict the delamination location and its extent. The simulation demonstrates that the wavelet neural network based on hybrid hierarchy genetic algorithm is robust, promising and converges very fast.     

GA-based fuzzy reinforcement learning for control of a magneticbearing system

This paper proposes a TD (temporal difference) and GA (genetic algorithm)-based reinforcement (TDGAR) learning method and applies it to the control of a real magnetic bearing system. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to perform the reinforcement learning task. The TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network to guide the learning of the other network (the action network) which determines the outputs (actions) of the TDGAR learning system. The action network can be a normal neural network or a neural fuzzy network. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. The key concept of the TDGAR learning scheme is to formulate the internal reinforcement signal as the fitness function for the GA such that the GA can evaluate the candidate solutions (chromosomes) regularly, even during periods without external feedback from the environment. This enables the GA to proceed to new generations regularly without waiting for the arrival of the external reinforcement signal. This can usually accelerate the GA learning since a reinforcement signal may only be available at a time long after a sequence of actions has occurred in the reinforcement learning problem. The proposed TDGAR learning system has been used to control an active magnetic bearing (AMB) system in practice. A systematic design procedure is developed to achieve successful integration of all the subsystems including magnetic suspension, mechanical structure, and controller training. The results show that the TDGAR learning scheme can successfully find a neural controller or a neural fuzzy controller for a self-designed magnetic bearing system.     

An optimizing method of RBF neural network based on genetic algorithm

In the traditional learning algorithms of radial basis function (RBF) neural network, the architecture of the network is hard to be decided; thereby, the learning ability and generalization ability are hard to achieve optimal. In this paper, we propose an algorithm to optimize the RBF neural network learning based on genetic algorithm; it uses hybrid encoding method, that is, encodes the network by binary encoding and encodes the weights by real encoding; the network architecture is self-adapted adjusted, and the weights are learned. Then, the network is further adjusted by pseudo inverse method or least mean square method. Experiments prove that the network gotten by this method has a better architecture and stronger classification ability, and the time of constructing the network artificially is saved. The algorithm is a self-adapted and intelligent learning algorithm.     

一种改进的仿生算法研究与分析

本文对仿生算法中的神经网络和遗传算法进行了分析和总结,针对神经网络方法训练速度慢,识别效率低,以及遗传算法选优时的过早收敛的问题,结合神经网络方法和遗传算法各自的特性,提出了相应的改进算法。在不破坏单个神经元的输入权值的基础上,采用数据预处理的方法来减少输入层的个数,从而提高进化学习的能力。     

A machine learning method for generation of a neural networkarchitecture: a continuous ID3 algorithm

The relation between the decision trees generated by a machine learning algorithm and the hidden layers of a neural network is described. A continuous ID3 algorithm is proposed that converts decision trees into hidden layers. The algorithm allows self-generation of a feedforward neural network architecture. In addition, it allows interpretation of the knowledge embedded in the generated connections and weights. A fast simulated annealing strategy, known as Cauchy training, is incorporated into the algorithm to escape from local minima. The performance of the algorithm is analyzed on spiral data.     

基于递阶遗传算法模糊加权神经网络的模糊规则自动获取

针对模糊规则的自动获取一直是模糊系统的一个瓶颈问题,提出一种基于递阶结构的混合编码遗传算法与进化规划相结合的模糊加权神经网络学习新算法,利用该算法同时优化模糊加权神经网络的结构和参数,最后说明了从网络中提取模糊规则的方法,从而自动获得最优的模糊规则。分析和实验结果表明,本文方法在规则提取和分类准确性等方面比其他方法更好。     

基于改进遗传算法的小波神经网络控制器的研究

小波神经网络是一种引入小波分析理论的前馈型神经网络,其与遗传算法的结合可以得到一种拥有良好全局优化搜索和良好局部时频特性的学习训练途径。本文提出了一种基于改进遗传算法的小波神经网络控制器,此方法可以克服基本遗传算法收敛速度慢,容易陷入＂早熟＂收敛,计算稳定性不好等一系列问题,进一步提高了小波神经网络控制器的性能。最后通过二级倒立摆仿真和实物控制,证明了控制器的有效性。     

基于粗糙集的进化神经网络入侵检测系统研究

针对目前实时入侵检测系统所处理的网络数据具有的非线性和高维的特点,提出基于粗糙集理论的进化神经网络入侵检测方法。对网络中截获的数据,利用粗糙集属性约简方法对其属性集进行约简,得到影响分类精度的重要属性。把约简后形成的训练样本进行数值化和归一化处理,作为神经网络的输入数据,再利用遗传算法较强的宏观搜索能力和全局寻优的特点,优化神经网络权值,并在此基础上进行神经网络学习,从而建立入侵检测系统的优化分析模型。实验结果表明,该算法学习速度快,有效提高了入侵检测系统的检测效率。     

Design of highly effective multilayer feedforward neural network by using genetic algorithm

This paper presents a highly effective and precise neural network method for choosing the activation functions (AFs) and tuning the learning parameters (LPs) of a multilayer feedforward neural network by using a genetic algorithm (GA). The performance of the neural network mainly depends on the learning algorithms and the network structure. The backpropagation learning algorithm is used for tuning the network connection weights, and the LPs are obtained by the GA to provide both fast and reliable learning. Also, the AFs of each neuron in the network are automatically chosen by a GA. The present study consists of 10 different functions to accomplish a better convergence of the desired input–output mapping. Test studies are performed to solve a set of two-dimensional regression problems for the proposed genetic-based neural network (GNN) and conventional neural network having sigmoid AFs and constant learning parameters. The proposed GNN has also been tested by applying it to three real problems in the fields of environment, medicine, and economics. Obtained results prove that the proposed GNN is more effective and reliable when compared with the classical neural network structure.     

A new genetic feature selection with neural network ensemble

A neural network ensemble is a learning paradigm in which a finite collection of neural networks is trained for the same task. Ensembles generally show better classification and generalization performance than a single neural network does. In this paper, a new feature selection method for a neural network ensemble is proposed for pattern classification. The proposed method selects an adequate feature subset for each constituent neural network of the ensemble using a genetic algorithm. Unlike the conventional feature selection method, each neural network is only allowed to have some (not all) of the considered features. The proposed method can therefore be applied to huge-scale feature classification problems. Experiments are performed with four databases to illustrate the performance of the proposed method.     

基于遗传优化的BP神经网络法在甲烷检测中的应用

针对传统的最小二乘法拟合红外传感器的输出特性曲线时存在误差大、计算复杂,传统的BP神经网络法拟合红外传感器的输出特性曲线时存在网络收敛速度慢、易陷入局部极小的问题,通过分析改进的最小二乘法和改进的基于遗传优化的BP神经网络法的拟合效果,指出改进的BP神经网络法拟合度较高,并给出了改进的BP神经网络法在甲烷体积分数检测中的实验结果。结果表明,该方法能够拟合出理想的曲线,有效提高了红外传感器的检测精度及响应速度。     

基于遗传算法优化投影寻踪技术的神经网络集成模型及其应用

首先利用遗传算法优化的投影寻踪技术对神经网络学习矩阵降维,再利用Bagging技术和不同的神经网络学习算法生成集成个体,并再次用遗传算法进化的投影寻踪技术对神经网络个体集成.建立基于遗传算法优化的投影寻踪技术神经网络集成模型,通过上证指数开盘价、收盘价进行实例分析,计算结果表明该方法具有较好的学习能力和泛化能力,在股市预测中预测精度高、稳定性好.     

一种有效的最优序参量重构方法

针对模式识别中协同方法存在的问题,提出了一种协同神经网络中序参量重构的方法,该方法是利用遗传算法的全局最优搜索能力,通过对训练样本集的学习,然后再通过在序参量的构建参数空间进行全局搜索来获得最优重构参数。利用实际采样得到的样本对新算法进行的测试表明,新方法确定能找到一组序参量重构参数,并能使识别性能有较大提高。     

基于权值变化的BP神经网络自适应学习率改进研究

针对传统神经网络的学习率由人为经验性设定,存在学习率设置过大或过小,容易导致无法收敛或收敛速度慢的问题,本文提出基于权值变化的自适应学习率改进方法,改善传统神经网络学习率受人为经验因素影响的弊端,提高误差精度,并结合正态分布模型与梯度上升法,提高收敛速度.本文以BP神经网络为例,对比固定学习率的神经网络,应用经典XOR问题仿真验证,结果表明本文的改进神经网络具有更快的收敛速度和更小的误差.     

Combining seasonal time series ARIMA method and neural networks with genetic algorithms for predicting the production value of the mechanical industry in Taiwan

Supplying industrial firms with an accurate method of forecasting the production value of the mechanical industry to facilitate decision makers in precise planning is highly desirable. Numerous methods, including the autoregressive integrated-moving average (ARIMA) model and artificial neural networks can make accurate forecasts based on historical data. The seasonal ARIMA (SARIMA) model and artificial neural networks can also handle data involving trends and seasonality. Although neural networks can make predictions, deciding the most appropriate input data, network structure and learning parameters are difficult. Therefore, this article presents a hybrid forecasting method that combines the SARIMA model and neural networks with genetic algorithms. Analytical results generated by the SARIMA model are inputted as the input data of a neural network. Subsequently, the number of neurons in the hidden layer and the number of learning parameters of the neural network architecture are globally optimized using genetic algorithms. This model is subsequently adopted to forecast seasonal time series data of the production value of the mechanical industry in Taiwan. The results presented here provide a valuable reference for decision makers in industry.     

NeC4.5: neural ensemble based C4.5

Decision tree is with good comprehensibility while neural network ensemble is with strong generalization ability. These merits are integrated into a novel decision tree algorithm NeC4.5. This algorithm trains a neural network ensemble at first. Then, the trained ensemble is employed to generate a new training set through replacing the desired class labels of the original training examples with those output from the trained ensemble. Some extra training examples are also generated from the trained ensemble and added to the new training set. Finally, a C4.5 decision tree is grown from the new training set. Since its learning results are decision trees, the comprehensibility of NeC4.5 is better than that of neural network ensemble. Moreover, experiments show that the generalization ability of NeC4.5 decision trees can be better than that of C4.5 decision trees.