Bounds of the incremental gain for discrete-time recurrent neural networks

As a nonlinear system, a recurrent neural network generally has an incremental gain different from its induced norm. While most of the previous research efforts were focused on the latter, this paper presents a method to compute an effective upper bound of the former for a class of discrete-time recurrent neural networks, which is not only applied to systems with arbitrary inputs but also extended to systems with small-norm inputs. The upper bound is computed by simple optimizations subject to linear matrix inequalities (LMIs). To demonstrate the wide connections of our results to problems in control, the servomechanism is then studied, where a feedforward neural network is designed to control the output of a recurrent neural network to track a set of trajectories. This problem can be converted into the synthesis of feedforward-feedback gains such that the incremental gain of a certain system is minimized. An algorithm to perform such a synthesis is proposed and illustrated with a numerical example.     

A context layered locally recurrent neural network for dynamic system identification

This paper puts forward a novel recurrent neural network (RNN), referred to as the context layered locally recurrent neural network (CLLRNN) for dynamic system identification. The CLLRNN is a dynamic neural network which appears in effective in the input–output identification of both linear and nonlinear dynamic systems. The CLLRNN is composed of one input layer, one or more hidden layers, one output layer, and also one context layer improving the ability of the network to capture the linear characteristics of the system being identified. Dynamic memory is provided by means of feedback connections from nodes in the first hidden layer to nodes in the context layer and in case of being two or more hidden layers, from nodes in a hidden layer to nodes in the preceding hidden layer. In addition to feedback connections, there are self-recurrent connections in all nodes of the context and hidden layers. A dynamic backpropagation algorithm with adaptive learning rate is derived to train the CLLRNN. To demonstrate the superior properties of the proposed architecture, it is applied to identify not only linear but also nonlinear dynamic systems. The efficiency of the proposed architecture is demonstrated by comparing the results to some existing recurrent networks and design configurations. In addition, performance of the CLLRNN is analyzed through an experimental application to a dc motor connected to a load to show practicability and effectiveness of the proposed neural network. Results of the experimental application are presented to make a quantitative comparison with an existing recurrent network in the literature.     

State estimation for delayed neural networks

In this letter, the state estimation problem is studied for neural networks with time-varying delays. The interconnection matrix and the activation functions are assumed to be norm-bounded. The problem addressed is to estimate the neuron states, through available output measurements, such that for all admissible time-delays, the dynamics of the estimation error is globally exponentially stable. An effective linear matrix inequality approach is developed to solve the neuron state estimation problem. In particular, we derive the conditions for the existence of the desired estimators for the delayed neural networks. We also parameterize the explicit expression of the set of desired estimators in terms of linear matrix inequalities (LMIs). Finally, it is shown that the main results can be easily extended to cope with the traditional stability analysis problem for delayed neural networks. Numerical examples are included to illustrate the applicability of the proposed design method.     

Neural-network control of nonaffine nonlinear system with zero dynamics by state and output feedback

This paper focuses on adaptive control of nonaffine nonlinear systems with zero dynamics using multilayer neural networks. Through neural network approximation, state feedback control is firstly investigated for nonaffine single-input-single-output (SISO) systems. By using a high gain observer to reconstruct the system states, an extension is made to output feedback neural-network control of nonaffine systems, whose states and time derivatives of the output are unavailable. It is shown that output tracking errors converge to adjustable neighborhoods of the origin for both state feedback and output feedback control.     

Recurrent neural networks for minimum infinity-norm kinematiccontrol of redundant manipulators

This paper presents two neural network approaches to minimum infinity-norm solution of the velocity inverse kinematics problem for redundant robots. Three recurrent neural networks are applied for determining a joint velocity vector with its maximum absolute value component being minimal among all possible joint velocity vectors corresponding to the desired end-effector velocity. In each proposed neural network approach, two cooperating recurrent neural networks are used. The first approach employs two Tank-Hopfield networks for linear programming. The second approach employs two two-layer recurrent neural networks for quadratic programming and linear programming, respectively. Both the minimal 2-norm and infinity-norm of joint velocity vector can be obtained from the output of the recurrent neural networks. Simulation results demonstrate that the proposed approaches are effective with the second approach being better in terms of accuracy and optimality     

A recurrent neural network for minimum infinity-norm kinematiccontrol of redundant manipulators with an improved problem formulationand reduced architecture complexity

This paper presents an improved neural computation where scheme for kinematic control of redundant manipulators based on infinity-norm joint velocity minimization. Compared with a previous neural network approach to minimum infinity-non kinematic control, the present approach is less complex in terms of cost of architecture. The recurrent neural network explicitly minimizes the maximum component of the joint velocity vector while tracking a desired end-effector trajectory. The end-effector velocity vector for a given task is fed into the neural network from its input and the minimum infinity-norm joint velocity vector is generated at its output instantaneously. Analytical results are given to substantiate the asymptotic stability of the recurrent neural network. The simulation results of a four-degree-of-freedom planar robot arm and a seven-degree-of-freedom industrial robot are presented to show the proposed neural network can effectively compute the minimum infinity-norm solution to redundant manipulators.     

Neural Net-Based H ∞ Control for a Class of Nonlinear Systems

A novel neural net-based approach for H _∞ control design of a class of nonlinear continuous-time systems is presented. In the proposed frameworks, the nonlinear system models are approximated by multilayer neural networks. The neural networks are piecewisely interpolated to generate a linear differential inclusion models by which a linear state feedback H _∞ control law can be constructed. It is shown that finding the permissible control gain matrices can be transformed to a standard linear matrix inequality problem and solved using the available computer software. This revised version was published online in June 2006 with corrections to the Cover Date.     

一类非线性系统的自组织模糊CMAC神经网络自适应重构跟踪控制

提出了一种非线性系统的自组织模糊CMAC(SOFCMAC)神经网络自适应重构跟踪控制方法,首先通过构造增广系统,设计出线性渐近跟踪控制器,然后采用SOFCMAC神经网络在线重构系统的非线性特性,以消除非线性特性引起的系统误差,可保证非线性系统闭环稳定并使系统输出跟踪期望输出.仿真算例证明了SOFCMAC神经网络自适应重构跟踪控制系统的稳定性.     

Modeling and sensitivity analysis of neural networks

This paper investigates the use of neural networks for the identification of linear time invariant dynamical systems. Two classes of networks, namely the multilayer feedforward network and the recurrent network with linear neurons, are studied. A notation based on Kronecker product and vector-valued function of matrix is introduced for neural models. It permits to write a feedforward network as a one step ahead predictor used in parameter estimation. A special attention is devoted to system theory interpretation of neural models. Sensitivity analysis can be formulated using derivatives based on the above-mentioned notation.     

A new approach to distributed fusion filtering for networked systems with random parameter matrices and correlated noises

This paper is concerned with the distributed filtering problem for a class of discrete-time stochastic systems over a sensor network with a given topology. The system presents the following main features: (i) random parameter matrices in both the state and observation equations are considered; and (ii) the process and measurement noises are one-step autocorrelated and two-step cross-correlated. The state estimation is performed in two stages. At the first stage, through an innovation approach, intermediate distributed least-squares linear filtering estimators are obtained at each sensor node by processing available output measurements not only from the sensor itself but also from its neighboring sensors according to the network topology. At the second stage, noting that at each sampling time not only the measurement but also an intermediate estimator is available at each sensor, attention is focused on the design of distributed filtering estimators as the least-squares matrix-weighted linear combination of the intermediate estimators within its neighborhood. The accuracy of both intermediate and distributed estimators, which is measured by the error covariance matrices, is examined by a numerical simulation example where a four-sensor network is considered. The example illustrates the applicability of the proposed results to a linear networked system with state-dependent multiplicative noise and different network-induced stochastic uncertainties in the measurements; more specifically, sensor gain degradation, missing measurements and multiplicative observation noises are considered as particular cases of the proposed observation model.     

Dynamic neural controllers for induction motor

The paper reports application of recently developed adaptive control techniques based on neural networks to the induction motor control. This case study represents one of the more difficult control problems due to the complex, nonlinear, and time-varying dynamics of the motor and unavailability of full-state measurements. A partial solution is first presented based on a single input-single output (SISO) algorithm employing static multilayer perceptron (MLP) networks. A novel technique is subsequently described which is based on a recurrent neural network employed as a dynamical model of the plant. Recent stability results for this algorithm are reported. The technique is applied to multiinput-multioutput (MIMO) control of the motor. A simulation study of both methods is presented. It is argued that appropriately structured recurrent neural networks can provide conveniently parameterized dynamic models for many nonlinear systems for use in adaptive control.     

A new recurrent neural-network architecture for visual patternrecognition

We propose a new type of recurrent neural-network architecture, in which each output unit is connected to itself and is also fully connected to other output units and all hidden units. The proposed recurrent neural network differs from Jordan's and Elman's recurrent neural networks with respect to function and architecture, because it has been originally extended from being a mere multilayer feedforward neural network, to improve discrimination and generalization powers. We also prove the convergence properties of the learning algorithm in the proposed recurrent neural network, and analyze the performance of the proposed recurrent neural network by performing recognition experiments with the totally unconstrained handwritten numeric database of Concordia University, Montreal, Canada. Experimental results have confirmed that the proposed recurrent neural network improves discrimination and generalization powers in the recognition of visual patterns     

Application of the OBD method for optimization of neural state variable estimators of the two-mass drive system

This paper presents neural estimators of the mechanical state variables of the electrical drive system with elastic joints. The non-measurable state variables, as the torsional torque and the load machine speed are estimated using multilayer feed-forward neural networks. The main stages of the design methodology of these neural estimators are presented. The optimal brain damage method is implemented for the structure optimization of each neural network. Then signals estimated by neural estimators are tested in the electrical drive control structure with additional feedbacks from the estimated shaft torque and the difference between the motor and the load speeds. The simulation results show good accuracy of both presented neural estimators for the wide range of changes of the reference speed and the load torque. The simulation results are then verified by laboratory experiments.     

Discrete-time delayed standard neural network model and its application

The research on the theory and application of artificial neural networks has achieved a great success over the past two decades. Recently, increasing attention has been paid to recurrent neural networks, which are rich in dynamics, highly parallelizable, and easily implementable with VLSI. Due to these attractive features, RNNs have widely been applied to system identification, control, optimization and associative memories[1]. Stability analysis, which is critical to any applications of R…     

State estimation of recurrent neural networks with time-varying delay: A novel delay partition approach

The state estimation problem is studied in this paper for a class of recurrent neural networks with time-varying delay. A novel delay partition approach is developed to derive a delay-dependent condition guaranteeing the existence of a desired state estimator for the delayed neural networks. The design of the gain matrix of the state estimator can be achieved by solving a linear matrix inequality, where no slack variable is involved. A numerical example is finally provided to show the advantage of the proposed approach over some existing results.     

Delay-Dependent Criteria for Global Robust Periodicity of Uncertain Switched Recurrent Neural Networks With Time-Varying Delay

In this paper, we introduce some ideas of switched systems into the field of neural networks and a large class of switched recurrent neural networks (SRNNs) with time-varying structured uncertainties and time-varying delay is investigated. Some delay-dependent robust periodicity criteria guaranteeing the existence, uniqueness, and global asymptotic stability of periodic solution for all admissible parametric uncertainties are devised by taking the relationship between the terms in the Leibniz-Newton formula into account. Because free weighting matrices are used to express this relationship and the appropriate ones are selected by means of linear matrix inequalities (LMIs), the criteria are less conservative than existing ones reported in the literature for delayed neural networks with parameter uncertainties. Some examples are given to show that the proposed criteria are effective and are an improvement over previous ones.     

Robust state estimation for perturbed systems with error variance and circular pole constraints: The discrete-time case

This paper is concerned with the problem of robust state estimation for linear perturbed discrete-time systems with error variance and circular pole constraints. The goal of this problem addressed is the design of a linear state estimator such that, for all admissible uncertainties in both state and output equations, the following two performance requirements are simultaneously satisfied: (1) the poles of the filtering matrix are all constrained to lie inside a prespecified circular region; and (2) the steady-state variance of the estimation error for each state is not more than the individual prespecified value. It is shown that this problem can be converted to an auxiliary matrix assignment problem and solved by using an algebraic matrix equation/inequality approach. Specifically, the conditions for the existence of desired estimators are obtained and the explicit expression of these estimators is also derived. The main results are then extended to the case when an H performance requirement is added. Finally, a numerical example is presented to demonstrate the significance of the proposed technique.     

Observers for linear multivariable systems with applications

This paper presents an algorithm for the design of asymptotic state estimators (observers) for index-invariant uniformly observable time-varying linear finite-dimensional multivariable systems. The results obtained indicate that asymptotic estimators can be employed in optimally designed regulators provided an increase from the optimal cost is tolerable. It is also shown that any uniformly observable and uniformly controllable plant with index-invariant observability and controllability matrices can be stabilized with an observer.     

Design of Asymptotic Estimators: An Approach Based on Neural Networks and Nonlinear Programming

A methodology to design state estimators for a class of nonlinear continuous-time dynamic systems that is based on neural networks and nonlinear programming is proposed. The estimator has the structure of a Luenberger observer with a linear gain and a parameterized (in general, nonlinear) function, whose argument is an innovation term representing the difference between the current measurement and its prediction. The problem of the estimator design consists in finding the values of the gain and of the parameters that guarantee the asymptotic stability of the estimation error. Toward this end, if a neural network is used to take on this function, the parameters (i.e., the neural weights) are chosen, together with the gain, by constraining the derivative of a quadratic Lyapunov function for the estimation error to be negative definite on a given compact set. It is proved that it is sufficient to impose the negative definiteness of such a derivative only on a suitably dense grid of sampling points. The gain is determined by solving a Lyapunov equation. The neural weights are searched for via nonlinear programming by minimizing a cost penalizing grid-point constraints that are not satisfied. Techniques based on low-discrepancy sequences are applied to deal with a small number of sampling points, and, hence, to reduce the computational burden required to optimize the parameters. Numerical results are reported and comparisons with those obtained by the extended Kalman filter are made     

A Neural-Network-Based Model for the Dynamic Simulation of the Tire/Suspension System While Traversing Road Irregularities

This paper deals with the simulation of the tire/suspension dynamics by using recurrent neural networks (RNNs). RNNs are derived from the multilayer feedforward neural networks, by adding feedback connections between output and input layers. The optimal network architecture derives from a parametric analysis based on the optimal tradeoff between network accuracy and size. The neural network can be trained with experimental data obtained in the laboratory from simulated road profiles (cleats). The results obtained from the neural network demonstrate good agreement with the experimental results over a wide range of operation conditions. The NN model can be effectively applied as a part of vehicle system model to accurately predict elastic bushings and tire dynamics behavior. Although the neural network model, as a black-box model, does not provide a good insight of the physical behavior of the tire/suspension system, it is a useful tool for assessing vehicle ride and noise, vibration, harshness (NVH) performance due to its good computational efficiency and accuracy.