Identification of nonlinear systems with outliers using wavelet neural networks based on annealing dynamical learning algorithm

This paper presents an annealing dynamical learning algorithm (ADLA) to train wavelet neural networks (WNNs) for identifying nonlinear systems with outliers. In ADLA–WNNs, wavelet-based support vector regression (WSVR) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs due to the similarity between WSVR and WNNs. After initialization, ADLA with nonlinear time-varying learning rates is applied to train the WNNs. In the ADLA, the determination of the learning rates would be a key work for the trade-off between stability and speed of convergence. A computationally efficient optimization method, particle swarm optimization (PSO), is adopted to find the optimal learning rates to overcome the stagnation in the training procedure of WNNs. Due to the advantages of WSVR and ADLA (WSVR–ADLA), the WSVR-based ADLA–WNNs (WSVR–ADLA–WNNs) can robust against outliers and achieve the promising efficiency of system identifications. Three examples are simulated to confirm the performance of the proposed algorithm. From the simulated results, the feasibility and superiority of the proposed WSVR–ADLA–WNNs for identifying nonlinear systems with artificial outliers are verified.     

特征选择和支持向量回归参数的联合优化

为提高支持向量回归算法的学习能力和泛化性能,提出了特征选择和支持向量回归参数的联合优化方法。联合优化方法采用主成分分析产生新的特征集,以方均误差为目标计算回归精度,并应用实数编码的免疫遗传算法求解此优化问题。仿真实验结果表明,联合优化的回归精度要优于单独优化特征和支持向量回归参数,而且优化速度更快。     

基于LSSVM的MIMO系统快速在线辨识方法

针对时变非线性多输入多输出(MIMO)系统在线辨识较困难的问题,提出一种基于最小二乘支持向量机(LSSVM)的快速在线辨识方法。介绍了现有LSSVM增量式和在线式学习算法,并为它引入了一些加速实现策略,得到LSSVM快速在线式学习算法。将MIMO系统分解为多个多输入单输出(MISO)子系统,对每一个MISO利用一个LSSVM在线建模;这些LSSVM执行快速在线式学习算法。数字仿真显示该方法建模速度快,模型预测精度高。     

Optimal PID Controller Autotuning Design for MIMO Nonlinear Systems Based on the Adaptive SLP Algorithm

In this paper, an adaptive swarm learning process (SLP) algorithm for designing the optimal proportional integral and derivative (PID) parameter for a multiple-input multiple-output (MIMO) control system is proposed. The SLP algorithm is proposed to improve the performance and convergence of PID parameter autotuning by applying the swarm algorithm and the learning process. The adaptive SLP algorithm improves the stability, performance and robustness of the traditional SLP algorithm to apply it to a MIMO control system. It can update the online weights of the SLP algorithm caused by the errors in the settling time, rise time and overshoot of the system based on a stable learning rate. The gradient descent is applied to update the weights. The stable learning rate is verified based on the Lyapunov stability theorem. Additionally, simulations are performed to verify the superiority of the algorithm in terms of performance and robustness. Results that compare the adaptive SLP algorithm with the traditional SLP, a neural network (NN), the genetic algorithm (GA), the particle swarm and optimization (PSO) algorithm and the kidney-inspired algorithm (KIA) based on a two-wheel inverted pendulum system are presented. With respect to performance and robustness, the adaptive SLP algorithm provides a better response than the traditional SLP, NN, GA, PSO and KIA.

     

Multiuser detection algorithm based on belief propagation in multiple-input multiple-output systems

In MIMO (multiple-input, multiple-output) systems, signals from differenttransmitting antennas interfere at each receiving antenna and multiuser detection (MUD)algorithms may be adopted to improve the system performance. This paper proposes anovel multiuser detection algorithm in MIMO systems based on the idea of "beliefpropagation" which has achieved great accomplishment in decoding of low-densityparity-check codes. The proposed algorithm has a low computation complexityproportional to the square of transmitting/receiving antenna number. Simulation resultsshow that under low signal-to-noise ratio (SNR) circumstances, the proposed algorithmoutperforms the traditional linear minimum mean square error (MMSE) detector while itencounters a "floor' of bit error rate under high SNR circumstances. So the proposedalgorithm is applicable to MIMO systems with channel coding and decoding. Although inthis paper the proposed algorithm is derived in MIMO systems, obviously it can be appliedto ordinary code-division m     

Neural‐network‐based predictive controller design: An application to temperature control of a plastic injection molding process

This paper presents a neural‐network‐based predictive control (NPC) method for a class of discrete‐time multi‐input multi‐output (MIMO) systems. A discrete‐time mathematical model using a recurrent neural network (RNN) is constructed and a learning algorithm adopting an adaptive learning rate (ALR) approach is employed to identify the unknown parameters in the recurrent neural network model (RNNM). The NPC controller is derived based on a modified predictive performance criterion, and its convergence is guaranteed by adopting an optimal algorithm with an adaptive optimal rate (AOR) approach. The stability analysis of the overall MIMO control system is well proven by the Lyapunov stability theory. A real‐time control algorithm is proposed which has been implemented using a digital signal processor, TMS320C31 from Texas Instruments. Two examples, including the control of a MIMO nonlinear system and the control of a plastic injection molding process, are used to demonstrate the effectiveness of the proposed strategy. Results from both numerical simulations and experiments show that the proposed method is capable of controlling MIMO systems with satisfactory tracking performance under setpoint and load changes. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

基于多维泰勒网的多入多出非线性时滞系统辨识

针对多入多出 （multiple input multiple output,MIMO） 非线性时滞系统辨识的准确性与实时性问题,提出基于多维泰勒网（multi-dimensional taylor network,MTN） 的辨识方案。MTN作为辨识模型,综合利用权剪枝 （weight-elimination,WE） 算法和共轭梯度（conjugate gradient,CG）算法,即WE-CG算法作为MTN辨识模型的学习算法;WE算法可以有效精简MTN辨识模型结构,从而降低计算复杂度、提高模型实时性能。最后,引入一个数值仿真例子和一个工程实例来验证所提辨识方案的有效性,同时与传统的MTN辨识方案作对比,给出了两者的准确性与复杂度分析,突出所提辨识方案的准确性与实时性。实验结果表明,所提方案够准确地对MIMO非线性时滞系统进行辨识。同时,相比传统的MTN辨识方案,所提辨识方案结构更精简,具有更低的算法复杂度。     

一种辨识Wiener-Hammerstein模型的新方法

针对非线性Wiener-Hammerstein模型,提出利用粒子群优化算法对非线性模型进行辨识的新方法.该方法的基本思想是将非线性系统的辨识问题转化为参数空间上的优化问题;然后采用粒子群优化算法获得该优化问题的解.为了进一步增强粒子群优化算法的辨识性能,提出利用一种混合粒子群优化算法.最后,仿真结果验证了该方法的有效性和可行性.     

On extreme learning machine for ε-insensitive regression in the primal by Newton method

In this paper, extreme learning machine (ELM) for ε-insensitive error loss function-based regression problem formulated in 2-norm as an unconstrained optimization problem in primal variables is proposed. Since the objective function of this unconstrained optimization problem is not twice differentiable, the popular generalized Hessian matrix and smoothing approaches are considered which lead to optimization problems whose solutions are determined using fast Newton–Armijo algorithm. The main advantage of the algorithm is that at each iteration, a system of linear equations is solved. By performing numerical experiments on a number of interesting synthetic and real-world datasets, the results of the proposed method are compared with that of ELM using additive and radial basis function hidden nodes and of support vector regression (SVR) using Gaussian kernel. Similar or better generalization performance of the proposed method on the test data in comparable computational time over ELM and SVR clearly illustrates its efficiency and applicability.     

求解非线性回归问题的Newton算法

针对大规模非线性回归问题,提出基于静态储备池的Newton算法.利用储备池搭建高维特征空间,将原始问题转化成与储备池维数相关的线性支持向量回归问题,并应用Newton算法求解.鲁棒损失函数的应用可抑制异常点对预测结果的干扰.通过与SVR(Support Vector Regression)及储备池Tikhonov正则化方法比较,验证了所提方法的快速性、较高的预测精度和较好的鲁棒性.     

多智能体粒子群优化的SVR 模型预测控制

参数的优化选择对支持向量回归机的预测精度和泛化能力影响显著,鉴于此,提出一种多智能体粒子群算法(MAPSO)寻优其参数的方法,并建立MAPSO支持向量回归模型,用于非线性系统的模型预测控制,推导出最优控制率.采用该算法对非线性系统进行仿真,并与基于粒子群算法、基于遗传算法优化支持向量回归机的模型预测控制方法和RBF神经网络的预测控制方法进行比较,结果表明,所提出的算法具有更好的控制性能,可以有效应用于非线性系统控制中.     

基于PLS和GAs的径基函数网络构造策略

鉴于传统径基函数网络(radial basis function network,简称RBFN)构造策略的不足,提出了基于偏最小二乘法(partial least squares,简称PLS)和遗传算法(genetic algorithms,简称GAs)的RBFN构造策略和一种更有效的径基宽度取值方法.在这个集成构造策略中,PLS克服了K-Means算法求取径基易陷入局部最优的弊病,并使合成径基比由正交算法获取的径基更具代表性;而所提出的径基宽度取值方法和GAs则为网络性能和结构的实质性改善与优化提供了保障.实验证实了基于PLS和GAs的RBFN构造策略及所提出的径基宽度取值方法的优越性、可靠性和有效性.     

一种多变量系统分散优化DMC算法

提出一种多变量系统的分散优化动态矩阵控制（ＤＭＣ）算法,该处冯动 经的特点,将多变量系统ＤＭＣ算法分散为若干单变量系统的ＤＭＣ算法,使多变量ＤＭＣ算法参数设计和算法求解计算人为简化。     

Dynamical pattern recognition for univariate time series and its application to an axial compressor

In this paper, a learning and recognition approach is proposed for univariate time series composed of output measurements of general nonlinear dynamical systems. Firstly, a class of dynamical systems in the canonical form is derived to describe the univariate time series by introducing coordinate transformation. An observer-based deterministic learning technique is then adopted to achieve dynamical modeling of the associated transformed systems of the training univariate time series, and the modeling results in the form of radial basis function network (RBFN) models are stored in a pattern library. Subsequently, multiple observer-based dynamical estimators containing the RBFN models in the pattern library are constructed for a test univariate time series, and a recognition decision scheme is proposed by the derived recognition indicator. On this basis, more concise recognition conditions are provided, which is beneficial for verifying the recognition results. Finally, simulation studies on the Rossler system and aero-engine stall warning verify the effectiveness of the proposed approach.     

A robust learning algorithm based on support vector regression and robust fuzzy cerebellar model articulation controller

For real-world applications, the obtained data are always subject to noise or outliers. The learning mechanism of cerebellar model articulation controller (CMAC), a neurological model, is to imitate the cerebellum of human being. CMAC has an attractive property of learning speed in which a small subset addressed by the input space determines output instantaneously. For fuzzy cerebellar model articulation controller (FCMAC), the concept of fuzzy is incorporated into CMAC to improve the accuracy problem. However, the distributions of errors into the addressed hypercubes may cause unacceptable learning performance for input data with noise or outliers. For robust fuzzy cerebellar model articulation controller (RFCMAC), the robust learning of M-estimator can be embedded into FCMAC to degrade noise or outliers. Meanwhile, support vector machine (SVR) is a machine learning theory based algorithm which has been applied successfully to a number of regression problems when noise or outliers exist. Unfortunately, the practical application of SVR is limited to defining a set of parameters for obtaining admirable performance by the user. In this paper, a robust learning algorithm based on support SVR and RFCMAC is proposed. The proposed algorithm has both the advantage of SVR, the ability to avoid corruption effects, and the advantage of RFCMAC, the ability to obtain attractive properties of learning performance and to increase accurate approximation. Additionally, particle swarm optimization (PSO) is applied to obtain the best parameters setting for SVR. From simulation results, it shows that the proposed algorithm outperforms other algorithms.     

基于ADQPSO-SVR的锅炉飞灰含碳量预测研究

针对锅炉飞灰含碳量的预测问题,提出了自适应扰动量子粒子群优化的支持向量回归机方法(ADQPSO-SVR),即在量子粒子群优化算法(QPSO)的基础上加入自适应扰动,克服了支持向量回归机(SVR)经验选择学习参数的弊端。用此改进算法对SVR的学习参数进行寻优,经过实例研究表明,ADQPSO算法的寻优能力较强,利用ADQPSO算法得到的SVR模型有较高的预测精度,同时与GA-BP算法和GA-RBF算法相比,ADQPSO-SVR能够提高锅炉飞灰含碳量预测的准确性及稳定性。     

Correlation analysis based MIMO neuro-fuzzy Hammerstein model with noises

A novel identification algorithm for neuro-fuzzy based MIMO Hammerstein system with noises by using the correlation analysis method is presented in this paper. A special test signal that contains independent separable signals and uniformly random multi-step signal is adopted to identify the MIMO Hammerstein system, resulting in the identification problem of the linear model separated from that of nonlinear part. As a result, it can circumvent the problem of initialization and convergence of the model parameters encountered by the existing iterative algorithms used for identification of MIMO Hammerstein model. Moreover, least square method based parameter identification algorithms of dynamic linear part and static nonlinear part are proposed to avoid the influence of noise. Examples are used to illustrate the effectiveness of the proposed method.     

A RECURSIVE 4SID FROM THE INPUT-OUTPUT POINT OF VIEW

In this paper, recursive algorithms of subspace state-space system identification (4SID) for multiple-input, multiple-output (MIMO), finite dimensional, linear time-invariant (FDLTI) systems are proposed. These algorithms are derived based on the Matrix Inversion Lemma. The investigation of our algorithms clarifies that a series of 4SID is the extension of the classical least square method to identification for multivariable systems, and also that our algorithms are the direct extension of the recursive least square algorithm to such ones. For PO-MOESP (the ordinary MOESP scheme with instrumental variables constructed from Past input and Output measurements), we show the mechanism of how the effect of the process and measurement noises is eliminated asymptotically by a projection related to the input and regressor matrices. “MOESP” is an abbreviation for “the MIMO output-error state-space model identification.”     

Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression

BackgroundShort-term load forecasting is an important issue that has been widely explored and examined with respect to the operation of power systems and commercial transactions in electricity markets. Of the existing forecasting models, support vector regression (SVR) has attracted much attention. While model selection, including feature selection and parameter optimization, plays an important role in short-term load forecasting using SVR, most previous studies have considered feature selection and parameter optimization as two separate tasks, which is detrimental to prediction performance.ObjectiveBy evolving feature selection and parameter optimization simultaneously, the main aims of this study are to make practitioners aware of the benefits of applying unified model selection in STLF using SVR and to provide one solution for model selection in the framework of memetic algorithm (MA).MethodsThis study proposes a comprehensive learning particle swarm optimization (CLPSO)-based memetic algorithm (CLPSO-MA) that evolves feature selection and parameter optimization simultaneously. In the proposed CLPSO-MA algorithm, CLPSO is applied to explore the solution space, while a problem-specific local search is proposed for conducting individual learning, thereby enhancing the exploitation of CLPSO.ResultsCompared with other well-established counterparts, benefits of the proposed unified model selection problem and the proposed CLPSO-MA for model selection are verified using two real-world electricity load datasets, which indicates the SVR equipped with CLPSO-MA can be a promising alternative for short-term load forecasting.     

Wavelet CMAC for MIMO uncertain nonlinear systems using a modified social ski-driver algorithm

This study uses a Mexican hat wavelet membership function for a cerebellar model articulation controller (CMAC) to develop a more efficient adaptive controller for multiple input multiple output (MIMO) uncertain nonlinear systems. The main controller is called the adaptive Mexican hat wavelet CMAC (MWCMAC), and an auxiliary controller is used to remove the residual error. For the MWCMAC, the online learning laws are derived from the gradient descent method. In addition, the learning rate values are very important and have a great impact on the performance of the control system; however, they are difficult to choose accurately. Therefore, a modified social ski driver (SSD) algorithm is proposed to find optimal learning rates for the control parameters. Finally, a magnetic ball levitation system and a nine-link biped robot are used to illustrate the effectiveness of the proposed SSD-based MWCMAC control system. The comparisons with other existing control algorithms have shown the superiority of the proposed control system.