Nonlinear system identification using memetic differential evolution trained neural networks

Several gradient-based approaches such as back propagation (BP) and Levenberg Marquardt (LM) methods have been developed for training the neural network (NN) based systems. But, for multimodal cost functions these procedures may lead to local minima, therefore, the evolutionary algorithms (EAs) based procedures are considered as promising alternatives. In this paper we focus on a memetic algorithm based approach for training the multilayer perceptron NN applied to nonlinear system identification. The proposed memetic algorithm is an alternative to gradient search methods, such as back-propagation and back-propagation with momentum which has inherent limitations of many local optima. Here we have proposed the identification of a nonlinear system using memetic differential evolution (DE) algorithm and compared the results with other six algorithms such as Back-propagation (BP), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), Differential Evolution (DE), Genetic Algorithm Back-propagation (GABP), Particle Swarm Optimization combined with Back-propagation (PSOBP). In the proposed system identification scheme, we have exploited DE to be hybridized with the back propagation algorithm, i.e. differential evolution back-propagation (DEBP) where the local search BP algorithm is used as an operator to DE. These algorithms have been tested on a standard benchmark problem for nonlinear system identification to prove their efficacy. First examples shows the comparison of different algorithms which proves that the proposed DEBP is having better identification capability in comparison to other. In example 2 good behavior of the identification method is tested on an one degree of freedom (1DOF) experimental aerodynamic test rig, a twin rotor multi-input-multi-output system (TRMS), finally it is applied to Box and Jenkins Gas furnace benchmark identification problem and its efficacy has been tested through correlation analysis.     

B-spline neural network design using improved differential evolution for identification of an experimental nonlinear process

B-Spline Neural Network (BSNN), a type of basis function neural network, is trained by gradient-based methods which may fall into local minima during the learning procedure. To overcome the limitations encountered by gradient-based optimization methods, we propose differential evolution (DE) – an evolutionary computation methodology – which can provide a stochastic search to adjust the control points of a BSNN. In this paper, we propose six DE approaches using chaotic sequences based on logistic mapping to train a BSNN. Chaos describes the complex behavior of a nonlinear deterministic system. The application of chaotic sequences instead of random sequences in DE is a powerful strategy to diversify the DE population and improve the DE's performance in preventing premature convergence to local minima. The numerical results presented here indicate that chaotic DE was effective for building a good BSNN model for the nonlinear identification of an experimental nonlinear yo–yo motion control system.     

An improved gravitational search algorithm for dynamic neural network identification

Gravitational search algorithm (GSA) is a newly developed and promising algorithm based on the law of gravity and interaction between masses. This paper proposes an improved gravitational search algorithm (IGSA) to improve the performance of the GSA, and first applies it to the field of dynamic neural network identification. The IGSA uses trial-and-error method to update the optimal agent during the whole search process. And in the late period of the search, it changes the orbit of the poor agent and searches the optimal agent’s position further using the coordinate descent method. For the experimental verification of the proposed algorithm, both GSA and IGSA are testified on a suite of four well-known benchmark functions and their complexities are compared. It is shown that IGSA has much better efficiency, optimization precision, convergence rate and robustness than GSA. Thereafter, the IGSA is applied to the nonlinear autoregressive exogenous (NARX) recurrent neural network identification for a magnetic levitation system. Compared with the system identification based on gravitational search algorithm neural network (GSANN) and other conventional methods like BPNN and GANN, the proposed algorithm shows the best performance.     

A neural differential evolution identification approach to nonlinear systems and modelling of shape memory alloy actuator

This paper proposes a hybrid modified differential evolution plus back‐propagation (MDE‐BP) algorithm to optimize the weights of the neural network model. In implementing the proposed training algorithm, the mutation phase of the differential evolution (DE) is modified by combining two mutation strategies rand/1 and best/1 to create trial vectors instead of only using one mutation operator or rand/1 or best/1 as the standard DE. The modification aims to balance the global exploration and local exploitation capacities of the algorithm in order to find potential global optimum solutions. Then the local searching ability of the back‐propagation (BP) algorithm is applied in that region so as to swiftly converge to the optimum solution. The performance and efficiency of the proposed method is tested by identifying some benchmark nonlinear systems and modeling the shape memory alloy actuator. The proposed training algorithm is compared with the other algorithms, such as the traditional DE and BP algorithm. As a result, the proposed method can improve the accuracy of the identification process.     

非线性动态系统的Wiener神经网络辨识法

提出了一种新的Wiener神经网络结构并将其应用于非线性动态系统辨识问题.首先,用Wiener模型对非线性动态系统进行描述,将其分解成线性动态子环节串接非线性静态增益的形式.其次,设计一种新型的神经网络结构,使网络权值对应于相应的Wiener模型参数;并推导了基于反向传播的网络权值调整方法.最后,通过网络迭代训练,可同时得到线性动态子环节和非线性静态增益的模型参数.通过一个Wiener模型的数值仿真来验证方法的有效性,仿真结果表明所提辨识方法切实可行.     

Adaptive output-feedback regulation for nonlinear delayed systems using neural network

A novel adaptive neural network （NN） output-feedback regulation algorithm for a class of nonlinear time-varying timedelay systems is proposed. Both the designed observer and controller are independent of time delay. Different from the existing results, where the upper bounding functions of time-delay terms are assumed to be known, we only use an NN to compensate for all unknown upper bounding functions without that assumption. The proposed design method is proved to be able to guarantee semi-global uniform ultimate boundedness of all the signals in the closed system, and the system output is proved to converge to a small neighborhood of the origin. The simulation results verify the effectiveness of the control scheme.     

基于Elman神经网络的非线性动态系统辨识

研究了应用动态递归神经网络实现动态系统辨识的原理和方法,在没有被辨识对象的先验知识情况下,通过改进的Elman网络实现了非线性动态系统的辨识。仿真结果表明,与前馈网络相比,Elman网络具有学习速度快、泛化能力强的特点,可用较小的网络结构实现高阶系统的辨识,适用于具有本质非线性动态系统的辨识。     

Differential evolution trained wavelet neural networks: Application to bankruptcy prediction in banks

In this study, differential evolution algorithm (DE) is proposed to train a wavelet neural network (WNN). The resulting network is named as differential evolution trained wavelet neural network (DEWNN). The efficacy of DEWNN is tested on bankruptcy prediction datasets viz. US banks, Turkish banks and Spanish banks. Further, its efficacy is also tested on benchmark datasets such as Iris, Wine and Wisconsin Breast Cancer. Moreover, Garson’s algorithm for feature selection in multi layer perceptron is adapted in the case of DEWNN. The performance of DEWNN is compared with that of threshold accepting trained wavelet neural network (TAWNN) [Vinay Kumar, K., Ravi, V., Mahil Carr, & Raj Kiran, N. (2008). Software cost estimation using wavelet neural networks. Journal of Systems and Software] and the original wavelet neural network (WNN) in the case of all data sets without feature selection and also in the case of four data sets where feature selection was performed. The whole experimentation is conducted using 10-fold cross validation method. Results show that soft computing hybrids viz., DEWNN and TAWNN outperformed the original WNN in terms of accuracy and sensitivity across all problems. Furthermore, DEWNN outscored TAWNN in terms of accuracy and sensitivity across all problems except Turkish banks dataset.     

非线性系统辨识方法研究

讨论了利用小波神经网络对非线性系统辨识的新方法。在辨识过程中,为了提高小波神经网络对非线性系统的辨识性能,使用一种改进粒子群优化算法对BP小波神经网络参数进行训练,求得最优值,达到对非线性系统辨识目的。在数值仿真中,与采用标准粒子群优化算法相比,结果显示了提出的方法在收敛性和稳定性等方面均得到了明显的改善。     

Nonlinear system identification using optimized dynamic neural network

In this paper, both off-line architecture optimization and on-line adaptation have been developed for a dynamic neural network (DNN) in nonlinear system identification. In the off-line architecture optimization, a new effective encoding scheme—Direct Matrix Mapping Encoding (DMME) method is proposed to represent the structure of neural network by establishing connection matrices. A series of GA operations are applied to the connection matrices to find the optimal number of neurons on each hidden layer and interconnection between two neighboring layers of DNN. The hybrid training is adopted to evolve the architecture, and to tune the weights and input delays of DNN by combining GA with the modified adaptation laws. The modified adaptation laws are subsequently used to tune the input time delays, weights and linear parameters in the optimized DNN-based model in on-line nonlinear system identification. The effectiveness of the architecture optimization and adaptation is extensively tested by means of two nonlinear system identification examples.     

pH中和系统辨识中的人工神经网络应用

pH中和作为化工、生物、发电和污水处理中的一个重要过程,具有极强的非线性和不确定性,很难对其进行精确建模,因此,pH值的控制一直是工业过程控制中的一个难题。本文借鉴了计算机领域中神经网络(NN)在非线性系统建模中的显著作用,结合对pH中和过程机理的分析,建立了基于BP神经网络的辨识模型,对典型的pH中和过程系统辨识进行了仿真研究,并进行了相关试验。试验结果表明:神经网络在pH中和过程辨识中具有较高的辨识精度,有着广阔的应用前景。     

Differential Evolution and Levenberg Marquardt Trained Neural Network Scheme for Nonlinear System Identification

This paper proposes a new nonlinear system identification scheme using differential evolution (DE), neural network and Levenberg Marquardt algorithm (LM). Here, DE and LM in a combined framework are used to train a neural network for achieving better convergence of neural network weight optimization. A number of examples including a practical case-study have been considered for implementation of different system identification methods namely, only NN, DE+NN and DE+LM+NN. After, a series of simulation studies of these methods on the different nonlinear systems it has been confirmed that the proposed DE and LM trained NN approach to nonlinear system identification has yielded better identification results in terms of time of convergence and less identification error.     

基于改进差分进化算法的非线性系统模型参数辨识

针对非线性模型的参数估计寻优较为困难的问题,提出一种基于改进的差分进化算法的非线性系统模型参数辨识新方法。通过引入一个自适应变异率,随着迭代的进行自适应调整缩放因子,从而在初期保持种群多样性以避免早熟,并在后期逐步降低变异率,保留优良信息,避免最优解遭到破坏。交叉概率采用动态非线性增加的方法,提高了收敛速度。为了验证算法性能,针对几类典型的非线性模型参数辨识问题进行了仿真研究,并将其应用于一类发酵动力学模型参数的估计中。结果表明改进算法的参数辨识精度高,收敛速度也比较快,有效提高了模型建立的精度与效率,为解决实际系统中参数估计问题提供了一条可行的途径。     

自适应差分进化算法优化BP神经网络的时间序列预测

针对BP神经网络学习算法随机初始化连接权值和阈值易使模型陷入局部极小点的缺点,设计了一种自适应差分进化算法优化BP神经网络的混合算法。该混合算法中,差分进化算法采用自适应变异和交叉因子优化BP神经网络的初始权值和阈值,再用预寻优得到的初始权值和阈值训练BP神经网络得到最优的权值和阈值。首先对改进的自适应差分进化算法运用测试函数进行性能测试,然后用一个经典时间序列问题对提出的混合算法进行了检验,并与一般的神经网络、ARIMA预测模型及其它混合预测模型进行了对比,实验结果表明,本文提出的混合算法有效并且明显提高了预测精度。     

动态非线性连续时间系统的小波神经网络辨识

将小波神经网络应用于动态非线性连续时间系统的辨识, 同时为了使神经网络的训练达到全局最优和加速小波神经网络训练的收敛速度, 提出了信赖域算法, 并研究了信赖域算法的收敛性. 随后进行了算例仿真, 证明了所提辨识方法的有效性.     

不确定非线性系统的直接自适应神经网络控制

为解决一类带干扰的不确定非线性系统中存在的两类未知项——未知函数和外界干扰,采用了直接自适应神经网络控制方法设计控制器。控制器设计中利用径向基函数神经网络良好的逼近性来近似未知函数,利用非线性衰减项来抑制干扰。所用方法结构简单、算法简洁,在一定条件下稳定性和收敛性能定性地得到保证。最后,仿真结果证明了该方法是正确的。     

LM-BP神经网络在大坝变形预测中的应用

为了对大坝进行切实有效的监控,需要建立一个良好的大坝预测模型。针对传统BP(Back-Propagation)神经网络存在的收敛速度慢和泛化能力弱等缺陷,利用LM-BP(Levenberg Marquardt Back Propagation)算法对大坝变形进行预测,并根据丹江口大坝1996和1997两年的变形观测数据,对大坝挠度预测结果进行分析。结果表明,所建立的LM-BP神经网络的预测精度和收敛速度明显提高。     

Adaptive consensus protocol for networks of multiple agents with nonlinear dynamics using neural networks

In this paper, an adaptive protocol is proposed to solve the consensus problem of multi‐agent systems with high‐order nonlinear dynamics by using neural networks (NNs) to approximate the unknown nonlinear system functions. It is derived that all agents achieve consensus if the undirected interaction graph is connected, and the transient performance of the multi‐agent system is also investigated. It shows that the adaptive protocol and the consensus analysis can be easily extended to switching networks by the existing LaSalle's Invariance Principle of switched systems. A numerical simulation illustrates the effectiveness of the proposed consensus protocol. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

滑动数据窗口驱动的贝叶斯-高斯网络及其在非线性系统辨识中的应用

工业控制场合中,需要获取非线性被控对象的结构特性,而系统动态响应的数据直接从外部特征上反映了非线性系统结构关系.为了充分利用非线性动态系统响应过程中的数据,本文提出了一种基于滑动数据窗口 (sliding data window)的贝叶斯-高斯神经网络 (SW-BGNN)模型.该模型将数据融合于网络模型结构中,借助于贝叶斯推理和高斯假设,利用滑动窗口数据,实现非线性动态系统的辨识和预测.整个SW-BGNN本身需要确定的参数很少,因此运算的时间很短,适合于非线性动态系统的在线辨识.将SW-BGNN应用于几个非线性动态系统的辨识和预测,仿真试验结果表明了SW-BGNN模型的有效性.     

Development of a coupled wavelet transform and evolutionary Levenberg‐Marquardt neural networks for hydrological process modeling

This research aims to present a general framework by which the most appropriate wavelet parameters including mother wavelet, vanishing moment, and decomposition level can be chosen for a joint wavelet transform and machine learning model. This study is organized in 2 parts: the first part presents an evolutionary Levenberg‐Marquardt neural network (ELMNN) model as the most effective machine learning configuration, and the second part describes how the wavelet transform can be effectively embedded with the developed ELMNN model. In this research, the rainfall and runoff time series data of 2 distinct watersheds at 2 different time scales (daily and monthly) were used to build the proposed hybrid wavelet transform and ELMNN model. The conclusions of this study showed that the Daubechies wavelet more than other wavelet families is capable to extract the informative features of hydrologic series. The vanishing moment and decomposition level of this mother wavelet should be selected based on the watershed behavior and the time resolution of rainfall and runoff time series, respectively. The verification results for both watersheds at daily and monthly time scales indicated root mean square error, peak value criterion, low value criterion, and Kling‐Gupta efficiency as about 0.017, 0.021, 0.023, and 0.91, respectively.