ANN-DT: an algorithm for extraction of decision trees fromartificial neural networks

Although artificial neural networks can represent a variety of complex systems with a high degree of accuracy, these connectionist models are difficult to interpret. This significantly limits the applicability of neural networks in practice, especially where a premium is placed on the comprehensibility or reliability of systems. A novel artificial neural-network decision tree algorithm (ANN-DT) is therefore proposed, which extracts binary decision trees from a trained neural network. The ANN-DT algorithm uses the neural network to generate outputs for samples interpolated from the training data set. In contrast to existing techniques, ANN-DT can extract rules from feedforward neural networks with continuous outputs. These rules are extracted from the neural network without making assumptions about the internal structure of the neural network or the features of the data. A novel attribute selection criterion based on a significance analysis of the variables on the neural-network output is examined. It is shown to have significant benefits in certain cases when compared with the standard criteria of minimum weighted variance over the branches. In three case studies the ANN-DT algorithm compared favorably with CART, a standard decision tree algorithm.     

前向神经网络合理隐含层结点个数估计

合理选择隐含层神经元个数是前向神经网络构造中的一个关键问题,对网络的泛化能力、训练速度等都具有重要的影响。该文提出了基于隐含层神经元输出之间的相关分析而进行隐含层神经元合理个数的估计方法,首先建立了基于网络输出和基于网络输出对网络各输入一阶偏导数的隐含层各神经元输出之间的相关程度度量,进而给出了基于模糊等价关系分析的神经元合理个数估计方法。具体应用结果证明了所提出方法的有效性。     

过程神经元网络及其在时变信息处理中的应用

针对时变信息处理和动态系统建模等类问题,建立了输入输出均为时变函数的过程神经元网络和有理式过程神经元网络2种网络模型.在输入输出为时变函数的过程神经元网络中,过程神经元的时间累积算子取为对时间的积分或其他代数运算,它的时空聚合机制和激励能同时反映外部时变输入信号对输出结果的空间聚合作用和时间累积效应,可实现非线性系统输入、输出之间的复杂映射关系.在有理式过程神经元网络中,其基本信息处理单元为由2个成对偶出现的过程神经元组成,逻辑上分为分子和分母2部分,通过有理式整合后输出,可有效提高过程神经元网络对带有奇异值过程函数的柔韧逼近性和在奇异值点附近反应的灵敏性.分析了2种过程神经元网络模型的性质,给出了具体学习算法,并以油田开发过程模拟和旋转机械故障诊断问题为例,验证了这2种网络模型在时变信息处理中的有效性.     

基于人工神经网络铂电阻非线性估计方法的研究􀀁

本文提出了一种基于人工神经网络的铂电阻传感器非线性估计方法,该方法用二次幂级数多项式拟合温度传感器的非线性模型,多项式的系数可由神经网络学习算法得到,当条件发生变化时,只要给出几组测量数据对,通过该方法可自动重新训练网络,获得新的多项式系数,实现传感器的非线性估计。     

Artificial neural networks in process estimation and control

In this contribution, the suitability of the artificial neural network methodology for solving some process engineering problems is discussed. First the concepts involved in the formulation of artificial neural networks are presented. Next the suitability of the technique to provide estimates of difficult to measure quality variables is demonstrated by application to industrial data. Measurements from established instruments are used as secondary variables for estimation of the “primary” quality variables. The advantage of using these estimates for feedback control is then demonstrated. The possibility of using neural network models directly within a model-based predictive control strategy is also considered, making use of an on-line optimization routine to determine the future inputs that will minimize the deviations between the desired and predicted outputs. Control is implemented in a receding horizon fashion. Application of the predictive controller to a nonlinear distillation system is used to indicate the potential of the neural network based control philosophy.     

Iterative inversion of fuzzified neural networks

The inversion of a neural network is a process of computing inputs that produce a given target when fed into the neural network. The inversion algorithm of crisp neural networks is based on the gradient descent search in which a candidate inverse is iteratively refined to decrease the error between its output and the target. In this paper. we derive an inversion algorithm of fuzzified neural networks from that of crisp neural networks. First, we present a framework of learning algorithms of fuzzified neural networks and introduce the idea of adjusting schemes for fuzzy variables. Next, we derive the inversion algorithm of fuzzified neural networks by applying the adjusting scheme for fuzzy variables to total inputs in the input layer. Finally, we make three experiments on the parity-three problem, examine the effect of the size of training sets on the inversion, and investigate how the fuzziness of inputs and targets of training sets affects the inversion     

Comparative study on local and global strategies for confidence estimation in neural networks and extensions to improve their predictive power

The use of confidence estimation techniques on neural networks outputs plays an important role when these mathematical models are applied in many practical applications. In general, the method to provide confidence estimation is dependent on the neural network architecture, but traditionally, most popular prediction interval (PI) estimation methods are only valid under strong assumptions, which are rarely satisfied in practical real problems. In this paper, we present a comparative study of local and global strategies for PI calculations and propose novel methods in both approaches to improve the predictive power of multilayer perceptron and radial basis function neural network models when the data are heterogeneous, both in density and residual variance. We apply our methods and make comparisons in a variety of simulated and real problems.     

Estimation of all-terminal network reliability using an artificial neural network

The exact calculation of all-terminal network reliability is an NP-hard problem, with computational effort growing exponentially with the number of nodes and links in the network. During optimal network design, a huge number of candidate topologies are typically examined with each requiring a network reliability calculation. Because of the impracticality of calculating all-terminal network reliability for networks of moderate to large size, Monte Carlo simulation methods to estimate network reliability and upper and lower bounds to bound reliability have been used as alternatives. This paper puts forth another alternative to the estimation of all-terminal network reliability — that of artificial neural network (ANN) predictive models. Neural networks are constructed, trained and validated using the network topologies, the link reliabilities, and a network reliability upperbound as inputs and the exact network reliability as the target. A hierarchical approach is used: a general neural network screens all network topologies for reliability followed by a specialized neural network for highly reliable network designs. Both networks with identical link reliability and networks with varying link reliability are studied. Results, using a grouped cross-validation approach, show that the ANN approach yields more precise estimates than the upperbound, especially in the worst cases. Using the reliability estimation methods of the ANN, the upperbound and backtracking, optimal network design by simulated annealing is considered. Results show that the ANN regularly produces superior network designs at a reasonable computational cost.Scope and purposeAn important application area of operations research is the design of structures, products or systems where both technical and business aspects must be considered. One expanding design domain is the design of computer or communications networks. While cost is a prime consideration, reliability is equally important. A common reliability measure is all-terminal reliability, the probability that all nodes (computers or terminals) on the network can communicate with all others. Exact calculation of all-terminal reliability is an NP-hard problem, precluding its use during optimal network topology design, where this calculation must be made thousands or millions of times. This paper presents a novel computationally practical method for estimating all-terminal network reliability. Is shown how a neural network can be used to estimate all-terminal network reliability by using the network topology, the link reliabilities and an upperbound on all-terminal network reliability as inputs. The neural network is trained and validated on a very minute fraction of possible network topologies, and once trained, it can be used without restriction during network design for a topology of a fixed number of nodes. The trained neural network is extremely fast computationally and can accommodate a variety of network design problems. The neural network approach, an upper bound approach and an exact backtracking calculation are compared for network design using simulated annealing for optimization and show that the neural network approach yields superior designs at manageable computational cost.     

基于序列输入的神经网络模型算法及应用

为提高神经网络的逼近能力,提出一种基于序列输入的神经网络模型及算法。模型隐层为序列神经元,输出层为普通神经元。输入为多维离散序列,输出为普通实值向量。先将各维离散输入序列值按序逐点加权映射,再将这些映射结果加权聚合之后映射为隐层序列神经元的输出,最后计算网络输出。采用Levenberg-Marquardt算法设计了该模型学习算法。仿真结果表明,当输入节点和序列长度比较接近时,模型的逼近能力明显优于普通神经网络。     

基于神经网络的故障飞机仿真

传统气动系数模型中，拟合法精度较差，插值法计算速度慢，且占内存多。利用神经网络一致逼近任意非线性连续函数的特性，训练具有一个三输入六输出的神经网络模型，建立故障飞机仿真系统。仿真结果和故障飞机自修复应用表明，文中所采用的神经网络建模方法是可行的。在自修复飞行控制系统研究中，为故障飞机建模所需大量故障状态气动系数数据处理提供一种新思路。     

Forecasting time series with a new architecture for polynomial artificial neural network

Polynomial artificial neural networks (PANN) have been shown to be powerful for forecasting nonlinear time series. The training time is small compared to the time used by other algorithms of artificial neural networks and the capacity to compute relations between the inputs and outputs represented by every term of the polynomial. In this paper a new structure of polynomial is presented that improves the performance of this type of network considering only non-integers exponents. The architecture adaptation uses genetic algorithm (GA) to find the optimal architecture for every example. Some examples of sunspots and chaotic time series are presented.     

Discrete probability estimation for classification usingcertainty-factor-based neural networks

Traditional probability estimation often demands a large amount of data for a problem of industrial scale. Neural networks have been used as an effective alternative for estimating input-output probabilities. In this paper, the certainty-factor-based neural network (CFNet) is explored for probability estimation in discrete domains. A new analysis presented here shows that the basis functions learned by the CFNet can bear precise semantics for dependencies. In the simulation study, the CFNet outperforms both the backpropagation network and the system based on the Rademacher-Walsh expansion. In the real-data experiments on splice junction and breast cancer data sets, the CFNet outperforms other neural networks and symbolic systems.     

Conditional probability density function estimation with sigmoidalneural networks

Previous developments in conditional density estimation have used neural nets to estimate statistics of the distribution or the marginal or joint distributions of the input-output variables. We modify the joint distribution estimating sigmoidal neural network to estimate the conditional distribution. Thus, the probability density of the output conditioned on the inputs is estimated using a neural network. We derive and implement the learning laws to train the network. We show that this network has computational advantages over a brute force ratio of joint and marginal distributions. We also compare its performance to a kernel conditional density estimator in a larger scale (higher dimensional) problem simulating more realistic conditions.     

基于粗糙集-神经网络集成的故障诊断

综合粗糙集和神经网络的优点,提出一种基于粗糙集-神经网络集成的智能故障诊断模型.在数据采集和预处理的基础上,利用粗糙集(RS)理论对原始故障诊断样本进行离散化处理,并根据条件属性(集)对决策属性的正域的大小来选择属性,提取出对诊断故障贡献最大的最小故障特征子集,从而确定神经网络的拓扑结构;通过网络训练建立故障特征与故障之间的映射关系,采用神经网络集成的方法实现故障的诊断.通过热电厂发电机组的故障诊断实例,表明了这种故障诊断方法的工程有效性.     

前向代数神经网络的函数逼近理论及学习算法

文中对ＭＰ神经元模型进行了推广,定义了多项代数神经元、多项式代数神经网络,将多项式代数融入代数神经网络,分析了前向多项式代数神经网络函数逼近能力及理论依据,设计出了一类双输入单输出的前向４层多层式代数神经网络模型,由该模型构成的网络能够逼近于给定的二元多项式到预定的精度。给出了在Ｐ－ａｄｉｃ意义下的多项式代数神经网络函数逼近整体学习算法,在学习的过程中,不存在局部极小,通过实例表明,该算法有效,最     

带有输出约束条件的随机多智能体系统容错控制

在有向通讯拓扑图下,针对一类具有输出约束和执行器偏差增益故障的非严格反馈随机多智能体系统,提出一种自适应神经网络容错控制设计方案.采用神经网络逼近未知非线性函数,构造障碍李雅普诺夫函数处理系统的输出约束问题,以反步法和动态面技术为框架,结合Nussbaum函数设计自适应神经网络容错控制方法.基于李雅普诺夫稳定性理论,证明所有跟随者输出与领导者输出达到一致,闭环系统的所有信号依概率半全局一致最终有界且系统输出限制在给定紧集内.论文最后通过仿真实验验证所给出控制方案的有效性.     

侦破神经网络训练中矛盾样本的模糊数学方法

基于模糊数学率，提出了一个从神经网络训练样本中侦破矛盾样本的方法，建立了其数学模型，并用此方法对从工厂取得的测量数据样本进行了检查，找到了对训练及检验结果影响很大的矛盾样本，并对剔除矛盾样本前后建立的模型进行了对比分析，结果表明用此方法侦破出矛盾样本并加以剔除后以大大改善训练及检验的结果，由此建立的模型更为准确。     

A hybrid neural network and simulated annealing approach to the unit commitment problem

In this paper, the authors present an approach combining the feedforward neural network and the simulated annealing method to solve unit commitment, a mixed integer combinatorial optimisation problem in power system. The artificial neural network is used to determine the discrete variables corresponding to the state of each unit at each time interval. The simulated annealing method is used to generate the continuous variables corresponding to the power output of each unit and the production cost. The type of neural network used in this method is a multi-layer perceptron trained by the back-propagation algorithm. A set of load profiles as inputs and the corresponding unit commitment schedules as outputs (satisfying the minimum up–down, spinning reserve and crew constraints) are utilized to train the network. A method to generate the training patterns is also presented. The experimental result demonstrates that the proposed approach can solve unit commitment in a reduced computational time with an optimum generation schedule.     

Neural network payload estimation for adaptive robot control

A concept is proposed for utilizing artificial neural networks to enhance the high-speed tracking accuracy of robotic manipulators. Tracking accuracy is a function of the controller's ability to compensate for disturbances produced by dynamical interactions between the links. A model-based control algorithm uses a nominal model of those dynamical interactions to reduce the disturbances. The problem is how to provide accurate dynamics information to the controller in the presence of payload uncertainty and modeling error. Neural network payload estimation uses a series of artificial neural networks to recognize the payload variation associated with a degradation in tracking performance. The network outputs are combined with a knowledge of nominal dynamics to produce a computationally efficient direct form of adaptive control. The concept is validated through experimentation and analysis on the first three links of a PUMA-560 manipulator. A multilayer perceptron architecture with two hidden layers is used. Integration of the principles of neural network pattern recognition and model-based control produces a tracking algorithm with enhanced robustness to incomplete dynamic information. Tracking efficacy and applicability to robust control algorithms are discussed.     

Fuzzy auto-associative neural networks for principal component extraction of noisy data.

In this paper, we propose a fuzzy auto-associative neural network for principal component extraction. The objective function is based on reconstructing the inputs from the corresponding outputs of the auto-associative neural network. Unlike the traditional approaches, the proposed criterion is a fuzzy mean squared error. We prove that the proposed objective function is an appropriate fuzzy formulation of auto-associative neural network for principal component extraction. Simulations are given to show the performances of the proposed neural networks in comparison with the existing method.