ARRBFNs with SVR for prediction of chaotic time series with outliers

In this article, annealing robust radial basis function networks (ARRBFNs), which consist of a radial basis function network and a support vector regression (SVR), and an annealing robust learning algorithm (ARLA) are proposed for the prediction of chaotic time series with outliers. In order to overcome the initial structural problems of the proposed neural networks, the SVR is utilized to determine the number of hidden nodes, the initial parameters of the kernel, and the initial weights for the proposed ARRBFNs. Then the ARLA that can conquer the outliers is applied to tune the parameters of the kernel and the weights in the proposed ARRBFNs under the initial structure with SVR. The simulation results of Mackey-Glass time series show that the proposed approach with different SVRs can cope with outliers and give a fast learning speed. The results of the simulation are also given to demonstrate the validity of proposed method for chaotic time series with outliers.     

A fast identification algorithm with outliers under Box-Cox transformation-based annealing robust radial basis function networks

In this article, a Box-Cox transformation-based annealing robust radial basis function networks (ARRBFNs) is proposed for an identification algorithm with outliers. Firstly, a fixed Box-Cox transformation-based ARRBFN model with support vector regression (SVR) is derived to determine the initial structure. Secondly, the results of the SVR are used as the initial structure in the fixed Box-Cox transformation-based ARRBFNs for the identification algorithm with outliers. At the same time, an annealing robust learning algorithm (ARLA) is used as the learning algorithm for the fixed Box-Cox transformation-based ARRBFNs, and applied to adjust the parameters and weights. Hence, the fixed Box-Cox transformation-based ARRBFNs with an ARLA have a fast convergence speed for an identification algorithm with outliers. Finally, the proposed algorithm and its efficacy are demonstrated with an illustrative example in comparison with Box-Cox transformation-based radial basis function networks.     

Control of magnetic bearing systems via the Chebyshevpolynomial-based unified model (CPBUM) neural network

A Chebyshev polynomial-based unified model (CPBUM) neural network is introduced and applied to control a magnetic bearing systems. First, we show that the CPBUM neural network not only has the same capability of universal approximator, but also has faster learning speed than conventional feedforward/recurrent neural network. It turns out that the CPBUM neural network is more suitable in the design of controller than the conventional feedforward/recurrent neural network. Second, we propose the inverse system method, based on the CPBUM neural networks, to control a magnetic bearing system. The proposed controller has two structures; namely, off-line and on-line learning structures. We derive a new learning algorithm for each proposed structure. The experimental results show that the proposed neural network architecture provides a greater flexibility and better performance in controlling magnetic bearing systems.     

The annealing robust backpropagation (ARBP) learning algorithm

Multilayer feedforward neural networks are often referred to as universal approximators. Nevertheless, if the used training data are corrupted by large noise, such as outliers, traditional backpropagation learning schemes may not always come up with acceptable performance. Even though various robust learning algorithms have been proposed in the literature, those approaches still suffer from the initialization problem. In those robust learning algorithms, the so-called M-estimator is employed. For the M-estimation type of learning algorithms, the loss function is used to play the role in discriminating against outliers from the majority by degrading the effects of those outliers in learning. However, the loss function used in those algorithms may not correctly discriminate against those outliers. In the paper, the annealing robust backpropagation learning algorithm (ARBP) that adopts the annealing concept into the robust learning algorithms is proposed to deal with the problem of modeling under the existence of outliers. The proposed algorithm has been employed in various examples. Those results all demonstrated the superiority over other robust learning algorithms independent of outliers. In the paper, not only is the annealing concept adopted into the robust learning algorithms but also the annealing schedule k/t was found experimentally to achieve the best performance among other annealing schedules, where k is a constant and t is the epoch number.     

Adaptive annealing learning algorithm-based robust wavelet neural networks for function approximation with outliers

In this study, a robust wavelet neural network (WNN) is proposed to approximate functions with outliers. In the proposed methodology, firstly, support vector machine with wavelet kernel function (WSVM) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs. Then, an adaptive annealing learning algorithm (AALA) is adopted to accommodate the translations, the dilations, and the weights of the WNNs. In the learning procedure, the AALA is proposed to overcome the problems of initialization and the cut-off points in the robust learning algorithm. Hence, when an initial structure of the WNNs is determined by a support vector regression (SVR) approach, the WNNs with AALA (AALA-WNNs) have fast convergence speed and can robust against outliers. Two examples are simulated to verify the feasibility and efficiency of the proposed algorithm.     

Robust least squares twin support vector machine for human activity recognition

Human activity recognition is an active area of research in Computer Vision. One of the challenges of activity recognition system is the presence of noise between related activity classes along with high training and testing time complexity of the system. In this paper, we address these problems by introducing a Robust Least Squares Twin Support Vector Machine (RLS-TWSVM) algorithm. RLS-TWSVM handles the heteroscedastic noise and outliers present in activity recognition framework. Incremental RLS-TWSVM is proposed to speed up the training phase. Further, we introduce the hierarchical approach with RLS-TWSVM to deal with multi-category activity recognition problem. Computational comparisons of our proposed approach on four well-known activity recognition datasets along with real world machine learning benchmark datasets have been carried out. Experimental results show that our method is not only fast but, yields significantly better generalization performance and is robust in order to handle heteroscedastic noise and outliers.     

Robust Learning Algorithm Based on Iterative Least Median of Squares

Outliers and gross errors in training data sets can seriously deteriorate the performance of traditional supervised feedforward neural networks learning algorithms. This is why several learning methods, to some extent robust to outliers, have been proposed. In this paper we present a new robust learning algorithm based on the iterative Least Median of Squares, that outperforms some existing solutions in its accuracy or speed. We demonstrate how to minimise new non-differentiable performance function by a deterministic approximate method. Results of simulations and comparison with other learning methods are demonstrated. Improved robustness of our novel algorithm, for data sets with varying degrees of outliers, is shown.     

A robust learning algorithm based on support vector regression and robust fuzzy cerebellar model articulation controller

For real-world applications, the obtained data are always subject to noise or outliers. The learning mechanism of cerebellar model articulation controller (CMAC), a neurological model, is to imitate the cerebellum of human being. CMAC has an attractive property of learning speed in which a small subset addressed by the input space determines output instantaneously. For fuzzy cerebellar model articulation controller (FCMAC), the concept of fuzzy is incorporated into CMAC to improve the accuracy problem. However, the distributions of errors into the addressed hypercubes may cause unacceptable learning performance for input data with noise or outliers. For robust fuzzy cerebellar model articulation controller (RFCMAC), the robust learning of M-estimator can be embedded into FCMAC to degrade noise or outliers. Meanwhile, support vector machine (SVR) is a machine learning theory based algorithm which has been applied successfully to a number of regression problems when noise or outliers exist. Unfortunately, the practical application of SVR is limited to defining a set of parameters for obtaining admirable performance by the user. In this paper, a robust learning algorithm based on support SVR and RFCMAC is proposed. The proposed algorithm has both the advantage of SVR, the ability to avoid corruption effects, and the advantage of RFCMAC, the ability to obtain attractive properties of learning performance and to increase accurate approximation. Additionally, particle swarm optimization (PSO) is applied to obtain the best parameters setting for SVR. From simulation results, it shows that the proposed algorithm outperforms other algorithms.     

Bayesian robust linear dynamic system approach for dynamic process monitoring

In this paper, a Bayesian robust linear dynamic system approach is proposed for process modeling. Traditional linear dynamic system (LDS) constructed with Kalman filter is designed by Gaussian assumption which can be easily violated in non-Gaussian modeling situations, especially those with outliers. To deal with this issue, the conventional Gaussian-based Kalman filter is modified with heavy tailed Student's t-distribution so as to deal with the non-Gaussian noise and modeling outliers. Then, a variational Bayesian expectation maximization (VBEM) algorithm is developed for learning parameters of the robust linear dynamic system. For process monitoring, traditional monitoring scheme are discussed and the residual space monitoring mechanism has been improved. To explore the feasibility and effectiveness, the proposed method is applied for fault detection, with detailed comparative studies with several other methods through the Tennessee Eastman benchmark.     

基于神经网络的机器人自学习控制器

提出一种神经网络与PID控制相结合的机器人自学习控制器.为加快神经网络的 学习收敛性,研究了有效的优化学习算法.以两关节机器人为对象的仿真表明,该控制器使机 器人跟踪希望轨迹,其系统响应、跟踪精度和鲁棒性优于常规的控制策略.     

一种新的鲁棒子空间建模方法

针对传统子空间建模技术中存在的两个难点问题,即对训练数据中的噪音或局外点非常敏感和基于批处理方式的大尺度高维样本模型学习计算非常费时,提出了一种新的鲁棒子空间建模方法.该方法先利用基于双平方函数的鲁棒估计,基于梯度下降的学习规则和M-估计器来同时学习和估计线性模型的初始参数,自动分级检测出初始训练样本集中的样本级局外点和样本中的信号级局外点;然后利用鲁棒的增量学习来更新参数,获得可靠的子空间模型.实验证明,这种新的鲁棒子空间建模方法能有效处理不同类型的噪音数据,在学习亮度子空间模型时能有效解决亮度明显变化、遮挡、噪音污染等敏感问题,并且具有较快的学习速度.     

基于交叉覆盖算法的入侵检测

文章提出了一种应用人工神经网络进行入侵检测分类器设计的新方法,即采用多层前向网络的交叉覆盖算法进行入侵检测分类器的设计。该算法克服了传统BP算法的收敛速度慢,易陷入局部最小点的问题。实验结果表明,该分类器用于入侵检测,效果良好,学习速度快,分类准确率高,为实现入侵检测分类器提供了一条准确高效的途径。     

单层神经网络的快速学习算法研究

该文提出一种适用于单层神经网络(SNN)训练的新颖的广义误差函数,给出了 SNN新的快速学习算法(FLA).进一步提出了一种广义系统辨识模型,对FLA的收敛性进 行了理论分析.实验表明:文中给出的新FLA比Karayiannis的LFA具有更快的收敛速度.     

一种基于鲁棒估计的极限学习机方法

极限学习机(ELM)是一种单隐层前馈神经网络(single-hidden layer feedforward neural networks,SLFNs),它相较于传统神经网络算法来说结构简单,具有较快的学习速度和良好的泛化性能等优点。ELM的输出权值是由最小二乘法(least square,LE)计算得出,然而经典的LS估计的抗差能力较差,容易夸大离群点和噪声的影响,从而造成训练出的参数模型不准确甚至得到完全错误的结果。为了解决此问题,提出一种基于M估计的采用加权最小二乘方法来取代最小二乘法计算输出权值的鲁棒极限学习机算法(RBELM),通过对多个数据集进行回归和分类分析实验,结果表明,该方法能够有效降低异常值的影响,具有良好的抗差能力。     

模糊层次神经网络的学习算法

该文基于模糊层次神经网络模型,提出了模糊层次神经网络的比例学习算法。模拟结果表明利用该算法训练的模糊层次神经网络具有较好的非逻辑归纳能力和特征抽取能力,并且学习速度也大大加快。     

Recurrent neural networks and robust time series prediction

We propose a robust learning algorithm and apply it to recurrent neural networks. This algorithm is based on filtering outliers from the data and then estimating parameters from the filtered data. The filtering removes outliers from both the target function and the inputs of the neural network. The filtering is soft in that some outliers are neither completely rejected nor accepted. To show the need for robust recurrent networks, we compare the predictive ability of least squares estimated recurrent networks on synthetic data and on the Puget Power Electric Demand time series. These investigations result in a class of recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component. Conventional least squares methods of fitting NARMA(p,q) neural network models are shown to suffer a lack of robustness towards outliers. This sensitivity to outliers is demonstrated on both the synthetic and real data sets. Filtering the Puget Power Electric Demand time series is shown to automatically remove the outliers due to holidays. Neural networks trained on filtered data are then shown to give better predictions than neural networks trained on unfiltered time series.     

Identification of nonlinear systems with outliers using wavelet neural networks based on annealing dynamical learning algorithm

This paper presents an annealing dynamical learning algorithm (ADLA) to train wavelet neural networks (WNNs) for identifying nonlinear systems with outliers. In ADLA–WNNs, wavelet-based support vector regression (WSVR) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs due to the similarity between WSVR and WNNs. After initialization, ADLA with nonlinear time-varying learning rates is applied to train the WNNs. In the ADLA, the determination of the learning rates would be a key work for the trade-off between stability and speed of convergence. A computationally efficient optimization method, particle swarm optimization (PSO), is adopted to find the optimal learning rates to overcome the stagnation in the training procedure of WNNs. Due to the advantages of WSVR and ADLA (WSVR–ADLA), the WSVR-based ADLA–WNNs (WSVR–ADLA–WNNs) can robust against outliers and achieve the promising efficiency of system identifications. Three examples are simulated to confirm the performance of the proposed algorithm. From the simulated results, the feasibility and superiority of the proposed WSVR–ADLA–WNNs for identifying nonlinear systems with artificial outliers are verified.     

非线性系统神经网络辨识的鲁棒BP算法

讨论系统辨识神经网络算法的鲁棒性问题。通过构造新的动态鲁棒目标函数得到的ＲＢＰ算法，能不断估计逼近精度，自动将品质好的样本置于强化学习域，并能有效地抵抗噪声干扰。实验结果表明，该算法具有鲁棒性强、收敛快、计算方便等特点。     

On the optimal design of fuzzy neural networks with robust learningfor function approximation

A novel robust learning algorithm for optimizing fuzzy neural networks is proposed to address two important issues: how to reduce the outlier effects and how to optimize fuzzy neural networks, in the function approximation. This algorithm is able to reduce the outlier effects by cooperating with a conventional robust approach, and then to optimize fuzzy neural networks by determining the optimal learning rates which can minimize the next-step mean error at each iteration of our algorithm.     

Median-Pi artificial neural network for forecasting

Neural Computing and Applications - Datasets with outliers can be predicted with robust learning methods or robust artificial neural networks. In robust artificial neural networks, the...