A robust recurrent simultaneous perturbation stochastic approximation training algorithm for recurrent neural networks

Training of recurrent neural networks (RNNs) introduces considerable computational complexities due to the need for gradient evaluations. How to get fast convergence speed and low computational complexity remains a challenging and open topic. Besides, the transient response of learning process of RNNs is a critical issue, especially for online applications. Conventional RNN training algorithms such as the backpropagation through time and real-time recurrent learning have not adequately satisfied these requirements because they often suffer from slow convergence speed. If a large learning rate is chosen to improve performance, the training process may become unstable in terms of weight divergence. In this paper, a novel training algorithm of RNN, named robust recurrent simultaneous perturbation stochastic approximation (RRSPSA), is developed with a specially designed recurrent hybrid adaptive parameter and adaptive learning rates. RRSPSA is a powerful novel twin-engine simultaneous perturbation stochastic approximation (SPSA) type of RNN training algorithm. It utilizes three specially designed adaptive parameters to maximize training speed for a recurrent training signal while exhibiting certain weight convergence properties with only two objective function measurements as the original SPSA algorithm. The RRSPSA is proved with guaranteed weight convergence and system stability in the sense of Lyapunov function. Computer simulations were carried out to demonstrate applicability of the theoretical results.     

Adaptive stochastic approximation by the simultaneous perturbation method

Stochastic approximation (SA) has long been applied for problems of minimizing loss functions or root finding with noisy input information. As with all stochastic search algorithms, there are adjustable algorithm coefficients that must be specified, and that can have a profound effect on algorithm performance. It is known that choosing these coefficients according to an SA analog of the deterministic Newton-Raphson algorithm provides an optimal or near-optimal form of the algorithm. However, directly determining the required Hessian matrix (or Jacobian matrix for root finding) to achieve this algorithm form has often been difficult or impossible in practice. The paper presents a general adaptive SA algorithm that is based on a simple method for estimating the Hessian matrix, while concurrently estimating the primary parameters of interest. The approach applies in both the gradient-free optimization (Kiefer-Wolfowitz) and root-finding/stochastic gradient-based (Robbins-Monro) settings, and is based on the "simultaneous perturbation (SP)" idea introduced previously. The algorithm requires only a small number of loss function or gradient measurements per iteration-independent of the problem dimension-to adaptively estimate the Hessian and parameters of primary interest. Aside from introducing the adaptive SP approach, the paper presents practical implementation guidance, asymptotic theory, and a nontrivial numerical evaluation. Also included is a discussion and numerical analysis comparing the adaptive SP approach with the iterate-averaging approach to accelerated SA.     

Convergence of simultaneous perturbation stochastic approximation for nondifferentiable optimization

We consider simultaneous perturbation stochastic approximation for function minimization. The standard assumption for convergence is that the function be three times differentiable, although weaker assumptions have been used for special cases. However, all work that we are aware of at least requires differentiability. We relax the differentiability requirement and prove convergence using convex analysis.     

Identification of continuous-time Hammerstein systems by simultaneous perturbation stochastic approximation

This paper proposes an identification method for Hammerstein systems using simultaneous perturbation stochastic approximation (SPSA). Here, the structure of nonlinear subsystem is assumed to be unknown, while the structure of linear subsystem, such as the system order, is assumed to be available. The main advantage of the SPSA-based method is that it can be applied to identification of Hammerstein systems with less restrictive assumptions. In order to clarify this point, piecewise affine functions with a large number of parameters are adopted to approximate the unknown nonlinear subsystems. Furthermore, the linear subsystems are supposed to be described in continuous-time. Though this class of systems closely reflects the actual systems, there are few methods to identify such models. Hence, the SPSA-based method is utilized to identify the parameters in both linear and nonlinear subsystems simultaneously. The effectiveness of the proposed method is evaluated through several numerical examples. The results demonstrate that the proposed algorithm is useful to obtain accurate models, even for high-dimensional parameter identification.     

Optimal random perturbations for stochastic approximation using asimultaneous perturbation gradient approximation

The simultaneous perturbation stochastic approximation (SPSA) algorithm has attracted considerable attention for challenging optimization problems where it is difficult or impossible to obtain a direct gradient of the objective (say, loss) function. The approach is based on a highly efficient simultaneous perturbation approximation to the gradient based on loss function measurements. SPSA is based on picking a simultaneous perturbation (random) vector in a Monte Carlo fashion as part of generating the approximation to the gradient. This paper derives the optimal distribution for the Monte Carlo process. The objective is to minimize the mean square error of the estimate. The authors also consider maximization of the likelihood that the estimate be confined within a bounded symmetric region of the true parameter. The optimal distribution for the components of the simultaneous perturbation vector is found to be a symmetric Bernoulli in both cases. The authors end the paper with a numerical study related to the area of experiment design     

Vibration suppression control of a flexible arm using a nonlinear observer with simultaneous perturbation stochastic approximation

The main objective in this study is the vibrational control of a one-link flexible arm system. A variable structure system (VSS) nonlinear observer has been proposed in order to reduce the oscillation when controlling the angle-of the flexible arm. The parameters of the nonlinear observer are optimized using a modified version of the simultaneous perturbation stochastic approximation (SPSA) algorithm. The SPSA algorithm is especially useful when the number of parameters to be adjusted is large, and makes it possible to estimate them simultaneously. For the vibration and position control, a model reference sliding-mode control (MR-SMC) has been proposed. The MR-SMC parameters are also optimized using a modified version of the SPSA algorithm. The simulations show that vibrational control of a one-link flexible arm system can be achieved more efficiently using our method. Therefore, by applying the MR-SMC method to a nonlinear observer, we can improve the performance in this kind of model using our proposed SPSA algorithm, and we can determine the control parameters very easy and efficiently.     

Robust segment-based object tracking using generalized hyperplane approximation

Tracking based on gradient descent algorithm using image gradient is one of the popular object tracking method. However, it easily fails to track when illumination changes. Although several illumination invariant features have been proposed, applying the invariant feature to the gradient descent method is not easy because the invariant feature is represented as a non-linear function of image pixel values and its Jacobian cannot be calculated in a closed-form. To make it possible, we introduce the generalized hyperplane approximation technique and apply it to histogram of oriented gradient (HOG) feature, one of the well-known illumination invariant feature. In addition, we achieve partial occlusion invariance using image segments. The hyperplanes are calculated from training segment images obtained by perturbing the motion parameter around the target region. Then, it is used to map the difference in non-linear feature of image onto the increment of alignment parameters. This process is mathematically same to the gradient descent method. The information from each segment is integrated by a simple weighted linear combination with confidence weights of segments. Compared to the previous tracking algorithms, our method shows very fast and stable tracking results in experiments on several practical image sequences.     

FPGA implementation of a pulse density neural network with learning ability using simultaneous perturbation

Hardware realization is very important when considering wider applications of neural networks (NNs). In particular, hardware NNs with a learning ability are intriguing. In these networks, the learning scheme is of much interest, with the backpropagation method being widely used. A gradient type of learning rule is not easy to realize in an electronic system, since calculation of the gradients for all weights in the network is very difficult. More suitable is the simultaneous perturbation method, since the learning rule requires only forward operations of the network to modify weights unlike the backpropagation method. In addition, pulse density NN systems have some promising properties, as they are robust to noisy situations and can handle analog quantities based on the digital circuits. We describe a field-programmable gate array realization of a pulse density NN using the simultaneous perturbation method as the learning scheme. We confirm the viability of the design and the operation of the actual NN system through some examples.     

Noise removal in electroencephalogram signals using an artificial neural network based on the simultaneous perturbation method

Electroencephalogram (EEG) recordings often experience interference by different kinds of noise, including white, muscle and baseline, severely limiting its utility. Artificial neural networks (ANNs) are effective and powerful tools for removing interference from EEGs. Several methods have been developed, but ANNs appear to be the most effective for reducing muscle and baseline contamination, especially when the contamination is greater in amplitude than the brain signal. An ANN as a filter for EEG recordings is proposed in this paper, developing a novel framework for investigating and comparing the relative performance of an ANN incorporating real EEG recordings. This method is based on a growing ANN that optimized the number of nodes in the hidden layer and the coefficient matrices, which are optimized by the simultaneous perturbation method. The ANN improves the results obtained with the conventional EEG filtering techniques: wavelet, singular value decomposition, principal component analysis, adaptive filtering and independent components analysis. The system has been evaluated within a wide range of EEG signals. The present study introduces a new method of reducing all EEG interference signals in one step with low EEG distortion and high noise reduction.     

Performance analysis of model-free PID tuning of MIMO systems based on simultaneous perturbation stochastic approximation

This paper addresses the performance comparison of simultaneous perturbation stochastic approximation (SPSA) based methods for PID tuning of MIMO systems. Four typical SPSA based methods, which are one-measurement SPSA (1SPSA), two-measurement SPSA (2SPSA), Global SPSA (GSPSA) and Adaptive SPSA (ASPSA) are examined. Their performances are evaluated by extensive simulation for several controller design examples, in terms of the stability of the closed-loop system, tracking performance and computation time. In addition, the performance of the SPSA based methods are compared to the other stochastic optimization based approaches. It turns out that the GSPSA based algorithm is the most practical in terms of the stability and the tracking performance.     

全局自适应神经网络跟踪控制及逼近域确定

针对自适应神经网络跟踪控制问题,提出一种确定逼近域的方法.采用参考信号取代未知非线性函数中的系统输出,神经网络用于逼近以参考信号为输入的未知不确定项.可以利用参考信号的界预先确定神经网络逼近域,再采用自适应鲁棒方法处理由于函数输入置换所引起的另一类不确定项.所得到的闭环系统是全局稳定的.仿真实例说明了该控制方法的有效性.     

Fuzzy controller design by using neural network techniques

This paper investigates the relationship between the piecewise linear fuzzy controller (PLFC), in which the membership functions for fuzzy variables and the associated inference rules are all in piecewise linear forms, and a Gaussian potential function network based controller (GPFNC), in which the network output is a weighted summation of hidden responses from a series of Gaussian potential function units (GPFU's). Systematic procedures are proposed for transformation from a PLFC to its GPFNC counterpart, and vice versa. Based on these transformation principles, a series of systematic and feasible steps is presented for the design of an optimized PLFC (PLFC*) by using neural network techniques. In the design procedures, the simplified PLFC is used as the initial controller structure, then a GPFNC, which gives the approximate control response to the initially given PLFC, is found for further optimization. A neutralization process is used to demonstrate the feasibility and the potential applicability of these intelligent controllers on the regulation of highly nonlinear chemical processes     

Remarks on tracking method of neural network weight change for adaptive type neural network feedforward feedback controller

A cost function is useful for a confirmation of neural network controller learning performance, but, this confirmation may not be correct for neural networks. Previous papers proposed a tracking method of neural network weight change and simulated it on the application of both learning and adaptive type neural network direct controllers. This paper applies the tracking method to an adaptive type neural network feedforward feedback controller and simulates it. The simulation results confirm that a track of the neural network weight change is separated into two trajectories. They also discuss the relationship between the feedback gain of the feedback controller and the parameter determining the neural network learning speed. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Intelligent anti-skid brake controller using a neural network

The performance of current anti-skid brake controllers on aircraft becomes degraded due to the uncertain nature of the runway conditions. This paper presents the design of an intelligent anti-skid neural controller to overcome this problem. Their learning ability, nonlinear mapping ability and pattern-recognition capability are the features of neural networks that are ideally suited to the design of intelligent controllers. The controller described here identifies the runway condition from the aircraft-wheel responses, and modulates the brake torque for optimum braking. The proposed controller exhibits robustness under variations in brake characteristics and runway conditions. Simulation results confirm the satisfactory performance of the controller in adapting to changes in runway conditions.     

Remarks on the tracking method of neural network weight change for a learning-type neural network feed-forward feed-back controller

Although many neural network controllers have been proposed, we still have to tune several parameters of neural networks in order to obtain a better learning performance in practical applications. Our tracking method provides a new aspect of this tuning of neural network parameters. It has been applied to adaptive and learning-type neural network direct controllers, and an adaptive-type neural network feed-forward feed-back controller. This work applied it to a learning-type neural network feed-forward feed-back controller. Simulation results confirmed its usefulness, and we discuss a transformation of the track on a 2D plane to one-dimensional values.     

A neural network-based real-time robot tracking controller using position sensitive detectors

Abstract: A real-time visual servo tracking system for an industrial robot has been developed. Instead of a charge coupled device (CCD), a position sensitive detector (PSD) is used as the real-time vision sensor due to its fast response (the light position is transduced to analogue current). A neural network learns the complex association between the 3D object position and its sensor reading, and uses it to track that object, either moving or stationary. It also turns out that this scheme lends itself to a user-friendly way to teach workpaths for industrial robots. Furthermore, for real-time use of the neural net, an efficient neural network architecture has been developed based on the concept of input space partitioning and local learning. Real experiments indicate the system's characteristics of fast processing and learning as well as optimal usage of network resources.     

Robust adaptive neural network control for environmental boundary tracking by mobile robots

The paper addresses the problem of environmental boundary tracking for the nonholonomic mobile robot with uncertain dynamics and external disturbances. To do environmental boundary tracking, a reference velocity is designed for the nonholonomic mobile robot. In this paper, a radial basis function neural network (NN) is used to approximate a nonlinear function containing the uncertain model terms and the elements of the Hessian matrix of the environmental concentration function. Then, the NN approximator is combined with a robust control to construct a robust adaptive NN control for the mobile robot to track the desired environment boundary. It is proved that the tracking error can be guaranteed to converge to zero in the ultimate. Simulation results are presented to illustrate the stability of the robust adaptive control. Copyright © 2011 John Wiley & Sons, Ltd.     

Robust visual tracking based on convolutional neural network with extreme learning machine

Multimedia Tools and Applications - Recently, deep learning has attracted substantial attention as a promising solution to many problems in computer vision. Among various deep learning...     

Robust tracking control of a unicycle-type wheeled mobile manipulator using a hybrid sliding mode fuzzy neural network

This article presents a robust tracking controller for an uncertain mobile manipulator system. A rigid robotic arm is mounted on a wheeled mobile platform whose motion is subject to nonholonomic constraints. The sliding mode control (SMC) method is associated with the fuzzy neural network (FNN) to constitute a robust control scheme to cope with three types of system uncertainties; namely, external disturbances, modelling errors, and strong couplings in between the mobile platform and the onboard arm subsystems. All parameter adjustment rules for the proposed controller are derived from the Lyapunov theory such that the tracking error dynamics and the FNN weighting updates are ensured to be stable with uniform ultimate boundedness (UUB).     

Robust redesign of a neural network controller in the presence of unmodeled dynamics

This work presents a neural network control redesign, which achieves robust stabilization in the presence of unmodeled dynamics restricted to be input to output practically stable (IOpS), without requiring any prior knowledge on any bounding function. Moreover, the state of the unmodeled dynamics is permitted to go unbounded provided that the nominal system state and/or the control input also go unbounded. The neural network controller is equipped with a resetting strategy to deal with the problem of possible division by zero, which may appear since we consider unknown input vector fields with unknown signs. The uniform ultimate boundedness of the system output to an arbitrarily small set, plus the boundedness of all other signals in the closed-loop is guaranteed.