A learning result for continuous-time recurrent neural networks

The following learning problem is considered, for continuous-time recurrent neural networks having sigmoidal activation functions. Given a “black box” representing an unknown system, measurements of output derivatives are collected, for a set of randomly generated inputs, and a network is used to approximate the observed behavior. It is shown that the number of inputs needed for reliable generalization (the sample complexity of the learning problem) is upper bounded by an expression that grows polynomially with the dimension of the network and logarithmically with the number of output derivatives being matched.     

Stable reinforcement learning with recurrent neural networks

In this paper,we present a technique for ensuring the stability of a large class of adaptively controlled systems.We combine IQC models of both the controlled system and the controller with a method of filtering control parameter updates to ensure stable behavior of the controlled system under adaptation of the controller.We present a specific application to a system that uses recurrent neural networks adapted via reinforcement learning techniques.The work presented extends earlier works on stable reinforcement learning with neural networks.Specifically,we apply an improved IQC analysis for RNNs with time-varying weights and evaluate the approach on more complex control system.     

Uncertainty-aware soft sensor using Bayesian recurrent neural networks

Data-driven soft sensors have been widely used to measure key variables for industrial processes. Soft sensors using deep learning models have attracted considerable attention and shown superior predictive performance. However, if a soft sensor encounters an unexpected situation in inferring data or if noisy input data is used, the estimated value derived by a standard soft sensor using deep learning may at best be untrustworthy. This problem can be mitigated by expressing a degree of uncertainty about the trustworthiness of the estimated value produced by the soft sensor. To address this issue of uncertainty, we propose using an uncertainty-aware soft sensor that uses Bayesian recurrent neural networks (RNNs). The proposed soft sensor uses a RNN model as a backbone and is then trained using Bayesian techniques. The experimental results demonstrated that such an uncertainty-aware soft sensor increases the reliability of predictive uncertainty. In comparisons with a standard soft sensor, it shows a capability to use uncertainties for interval prediction without compromising predictive performance.     

Effect of complexity on learning ability of recurrent neural networks

This paper demonstrates that recurrent neural networks can be used effectively to estimate unknown, complicated nonlinear dynamics. The emphasis of this paper is on the distinguishable properties of dynamics at the edge of chaos, i.e., between ordered behavior and chaotic behavior. We introduce new stochastic parameters, defined as combinations of standard parameters, and reveal relations between these parameters and the complexity of the network dynamics by simulation experiments. We then propose a novel learning method whose core is to keep the complexity of the network dynamics to the dynamics phase which has been distinguished using formulations of the experimental relations. In this method, the standard parameters of neurons are changed by the core part and also according to the global error measure calculated by the well-known simple back-propagation algorithm. Some simulation studies show that the core part is effective for recurrent neural network learning, and suggest the existence of excellent learning ability at the edge of chaos. This work was presented, in part, at the Second International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1997     

Reinforcement learning of multiple tasks using a hierarchical CMAC architecture

A reinforcement learning approach based on modular function approximation is presented. Cerebellar Model Articulation Controller (CMAC) networks are incorporated in the Hierarchical Mixtures of Experts (HME) architecture and the resulting architecture is referred to as HME-CMAC. A computationally efficient on-line learning algorithm based on the Expectation Maximization (EM) algorithm is proposed in order to achieve fast function approximation with the HME-CMAC architecture.

The Compositional Q-Learning (CQ-L) framework establishes the relationship between the Q-values of composite tasks and those of elemental tasks in its decomposition. This framework is extended here to allow rewards in non-terminal states. An implementation of the extended CQ-L framework using the HME-CMAC architecture is used to perform task decomposition in a realistic simulation of a two-linked manipulator having non-linear dynamics. The context-dependent reinforcement learning achieved by adopting this approach has advantages over monolithic approaches in terms of speed of learning, storage requirements and the ability to cope with changing goals.     

Adjoint EKF learning in recurrent neural networks for nonlinear active noise control

In this paper, active noise control using recurrent neural networks is addressed. A new learning algorithm for recurrent neural networks based on Adjoint Extended Kalman Filter is developed for active noise control. The overall control structure for active noise control is constructed using two recurrent neural networks: the first neural network is used to model secondary path of active noise control while the second one is employed to generate control signal. Real-time experiment of the proposed algorithm using digital signal processor is carried-out to show the effectiveness of the method.     

Assigning discounts in a marketing campaign by using reinforcement learning and neural networks

In this work, RL is used to find an optimal policy for a marketing campaign. Data show a complex characterization of state and action spaces. Two approaches are proposed to circumvent this problem. The first approach is based on the self-organizing map (SOM), which is used to aggregate states. The second approach uses a multilayer perceptron (MLP) to carry out a regression of the action-value function. The results indicate that both approaches can improve a targeted marketing campaign. Moreover, the SOM approach allows an intuitive interpretation of the results, and the MLP approach yields robust results with generalization capabilities.     

Creating Advice-Taking Reinforcement Learners

Learning from reinforcements is a promising approach for creating intelligent agents. However, reinforcement learning usually requires a large number of training episodes. We present and evaluate a design that addresses this shortcoming by allowing a connectionist Q-learner to accept advice given, at any time and in a natural manner, by an external observer. In our approach, the advice-giver watches the learner and occasionally makes suggestions, expressed as instructions in a simple imperative programming language. Based on techniques from knowledge-based neural networks, we insert these programs directly into the agent's utility function. Subsequent reinforcement learning further integrates and refines the advice. We present empirical evidence that investigates several aspects of our approach and shows that, given good advice, a learner can achieve statistically significant gains in expected reward. A second experiment shows that advice improves the expected reward regardless of the stage of training at which it is given, while another study demonstrates that subsequent advice can result in further gains in reward. Finally, we present experimental results that indicate our method is more powerful than a naive technique for making use of advice.     

Robust dynamic surface control of flexible joint robots using recurrent neural networks

A robust neuro-adaptive controller for uncertain flexible joint robots is presented. This control scheme integrates H-infinity disturbance attenuation design and recurrent neural network adaptive control technique into the dynamic surface control framework. Two recurrent neural networks are used to adaptively learn the uncertain functions in a flexible joint robot. Then, the effects of approximation error and filter error on the tracking performance are attenuated to a prescribed level by the embedded H-infinity controller, so that the desired H-infinity tracking performance can be achieved. Finally, simulation results verify the effectiveness of the proposed control scheme.     

Exponential synchronization of memristor-based recurrent neural networks with time delays

In this paper, the synchronization control of a general class of memristor-based recurrent neural networks with time delays is investigated. A delay-dependent feedback controller is derived to achieve the exponential synchronization based on the drive-response concept, linear matrix inequalities (LMIs) and Lyapunov functional method. Finally, a numerical example is given to illustrate the derived theoretical results.     

Classical and superposed learning for quantum weightless neural networks

A supervised learning algorithm for quantum neural networks (QNN) based on a novel quantum neuron node implemented as a very simple quantum circuit is proposed and investigated. In contrast to the QNN published in the literature, the proposed model can perform both quantum learning and simulate the classical models. This is partly due to the neural model used elsewhere which has weights and non-linear activations functions. Here a quantum weightless neural network model is proposed as a quantisation of the classical weightless neural networks (WNN). The theoretical and practical results on WNN can be inherited by these quantum weightless neural networks (qWNN). In the quantum learning algorithm proposed here patterns of the training set are presented concurrently in superposition. This superposition-based learning algorithm (SLA) has computational cost polynomial on the number of patterns in the training set.     

Multiple models switching control based on recurrent neural networks

This paper presents a novel approach in designing adaptive controller to improve the transient performance for a class of nonlinear discrete-time systems under different operating modes. The proposed scheme consists of generalized minimum variance (GMV) controllers and a compensating controller. GMV controllers are based on the known nominal linear multiple models, while the compensating controller is based upon a recurrent neural network. The adaptation law of network weight is derived from Lyapunov stability theory. A suitable switching control strategy is applied to choose the best controller by the performance indices at every sampling instant. Simulations are discussed in order to illustrate the merits of the proposed method.     

Encoding subcomponents in cooperative co-evolutionary recurrent neural networks

Cooperative coevolution employs evolutionary algorithms to solve a high-dimensional search problem by decomposing it into low-dimensional subcomponents. Efficient problem decomposition methods or encoding schemes group interacting variables into separate subcomponents in order to solve them separately where possible. It is important to find out which encoding schemes efficiently group subcomponents and the nature of the neural network training problem in terms of the degree of non-separability. This paper introduces a novel encoding scheme in cooperative coevolution for training recurrent neural networks. The method is tested on grammatical inference problems. The results show that the proposed encoding scheme achieves better performance when compared to a previous encoding scheme.     

Call classification using recurrent neural networks, support vector machines and finite state automata

Our objective is spoken-language classification for helpdesk call routing using a scanning understanding and intelligent-system techniques. In particular, we examine simple recurrent networks, support-vector machines and finite-state transducers for their potential in this spoken-language-classification task and we describe an approach to classification of recorded operator-assistance telephone utterances. The main contribution of the paper is a comparison of a variety of techniques in the domain of call routing. Support-vector machines and transducers are shown to have some potential for spoken-language classification, but the performance of the neural networks indicates that a simple recurrent network performs best for helpdesk call routing. Sheila Garfield received a BSc (Hons) in computing from the University of Sunderland in 2000 where, as part of her programme of study, she completed a project associated with aphasic language processing. She received her PhD from the same university, in 2004, for a programme of work connected with hybrid intelligent systems and spoken-language processing. In her PhD thesis, she collaborated with British Telecom and suggested a novel hybrid system for call routing. Her research interests are natural language processing, hybrid systems, intelligent systems. Stefan Wermter holds the Chair in Intelligent Systems and is leading the Intelligent Systems Division at the University of Sunderland, UK. His research interests are intelligent systems, neural networks, cognitive neuroscience, hybrid systems, language processing and learning robots. He has a diploma from the University of Dortmund, Germany, an MSc from the University of Massachusetts, USA, and a PhD in habilitation from the University of Hamburg, Germany, all in Computer Science. He was a Research Scientist at Berkeley, CA, before joining the University of Sunderland. Professor Wermter has written edited, or contributed to 8 books and published about 80 articles on this research area.     

A learning result for continuous-time recurrent neural networks

The following learning problem is considered, for continuous-time recurrent neural networks having sigmoidal activation functions. Given a “black box” representing an unknown system, measurements of output derivatives are collected, for a set of randomly generated inputs, and a network is used to approximate the observed behavior. It is shown that the number of inputs needed for reliable generalization (the sample complexity of the learning problem) is upper bounded by an expression that grows polynomially with the dimension of the network and logarithmically with the number of output derivatives being matched.     

Continuous attractors of a class of recurrent neural networks

Recurrent neural networks (RNNs) may possess continuous attractors, a property that many brain theories have implicated in learning and memory. There is good evidence for continuous stimuli, such as orientation, moving direction, and the spatial location of objects could be encoded as continuous attractors in neural networks. The dynamical behaviors of continuous attractors are interesting properties of RNNs. This paper proposes studying the continuous attractors for a class of RNNs. In this network, the inhibition among neurons is realized through a kind of subtractive mechanism. It shows that if the synaptic connections are in Gaussian shape and other parameters are appropriately selected, the network can exactly realize continuous attractor dynamics. Conditions are derived to guarantee the validity of the selected parameters. Simulations are employed for illustration.     

Indirect adaptive control of nonlinear dynamic systems using self recurrent wavelet neural networks via adaptive learning rates

This paper proposes an indirect adaptive control method using self recurrent wavelet neural networks (SRWNNs) for dynamic systems. The architecture of the SRWNN is a modified model of the wavelet neural network (WNN). However, unlike the WNN, since a mother wavelet layer of the SRWNN is composed of self-feedback neurons, the SRWNN can store the past information of wavelets. In the proposed control architecture, two SRWNNs are used as both an identifier and a controller. The SRWNN identifier approximates dynamic systems and provides the SRWNN controller with information about the system sensitivity. The gradient-descent method using adaptive learning rates (ALRs) is applied to train all weights of the SRWNN. The ALRs are derived from discrete Lyapunov stability theorem, which are applied to guarantee the convergence of the proposed control system. Finally, we perform some simulations to verify the effectiveness of the proposed control scheme.     

递归神经网络的设计与应用

结合实例,给出了递归神经网络的完整设计步骤,包括网络结构的选定,学习算法的选择和网络参数的训练过程。重点研究了学习速率的初始值选取及其调整顺序。给出的递归网络的设计方法,可以适用于多种递归神经网络。     

The generation of animated motion by means of a recurrent neural network

The possibility to use neural networks to guide animated motion sequences is investigated. The performance of two recurrent architectures, both derived from the cascade-correlation network, is compared. These architectures only differ in the objective function used to train the hidden units. Small differences in performance were observed, but both networks could successfully produce simple motion sequences. An animation environment was created to display arm movement and walking sequences.     

Dynamic classification of ballistic missiles using neural networks and hidden Markov models

This paper addresses dynamic classification of different ranges of ballistic missiles (BM) for air defense application based on kinematic attributes acquired by radars for taking appropriate measures to intercept them. The problem of dynamic classification is formulated using real-time neural network (RTNN) and hidden Markov model (HMM). The idea behind these algorithms is to calculate the output in one pass rather than training and computing over large number of iterations. Besides, to meet the conflicting requirements of classifying small as well as long-range trajectories, we are also proposing a formulation for partitioning the trajectory by using moving window concept. This concept allows us to use parameters in localized frame which helps in handling wide-range of trajectories to fit into the same network. These algorithms are evaluated using the simulated data generated from 6 degree-of-freedom (6DOF) mathematical model, which models missile trajectories. Experimental results show that both the networks are classifying above 95% with real-time neural network outperforming HMM in terms of time of computation on same data. The small classification time enables the use of real-time classification neural network in complex scenario of multi-radar, multi-target engagement by interceptor missiles. To the best of our knowledge this is the first time an attempt is made to classify ballistic missiles using RTNN and HMM.