Asymptotic tracking by a reinforcement learning-based adaptive critic controller

Adaptive critic (AC) based controllers are typically discrete and/or yield a uniformly ultimately bounded stability result because of the presence of disturbances and unknown approximation errors. A continuous-time AC controller is developed that yields asymptotic tracking of a class of uncertain nonlinear systems with bounded disturbances. The proposed AC-based controller consists of two neural networks (NNs) – an action NN, also called the actor, which approximates the plant dynamics and generates appropriate control actions; and a critic NN, which evaluates the performance of the actor based on some performance index. The reinforcement signal from the critic is used to develop a composite weight tuning law for the action NN based on Lyapunov stability analysis. A recently developed robust feedback technique, robust integral of the sign of the error (RISE), is used in conjunction with the feedforward action neural network to yield a semiglobal asymptotic result. Experimental results are provided that illustrate the performance of the developed controller.     

Neural network-based asymptotic tracking control of unknown nonlinear systems with continuous control command

ABSTRACT

This paper proposes a robust tracking controller for a class of nonlinear second-order systems with time-varying uncertainties. The controller is mainly based on the robust integral of the sign of the error (RISE) control approach to achieve an asymptotic stability result with a continuous control command in the presence of additive uncertainties. An adaptive feedforward neural network control term is blended with a new RISE controller to improve the system's transient performance. The proposed RISE controller is a modified version of the existing saturated RISE controller such that only sign of the derivative of the output is needed. The stability of the closed-loop system is well studied, where a local asymptotic stability is proven. The controller performance is validated through simulations on a two-degree-of-freedom lower limb robotic exoskeleton.     

Iterative inversion of neural networks and its application toadaptive control

An iterative constrained inversion technique is used to find the control inputs to the plant. That is, rather than training a controller network and placing this network directly in the feedback or feedforward paths, the forward model of the plant is learned, and iterative inversion is performed on line to generate control commands. The control approach allows the controllers to respond online to changes in the plant dynamics. This approach also attempts to avoid the difficulty of analysis introduced by most current neural network controllers, which place the highly nonlinear neural network directly in the feedback path. A neural network-based model reference adaptive controller is also proposed for systems having significant dynamics between the control inputs and the observed (or desired) outputs and is demonstrated on a simple linear control system. These results are interpreted in terms of the need for a dither signal for on-line identification of dynamic systems.     

Compound control of a compound cosine function neural network and PD for manipulators

In this paper, a compound cosine function neural network controller for manipulators is presented based on the combination of a cosine function and a unipolar sigmoid function. The compound control scheme based on a proportional-differential (PD) feedback control plus the cosine function neural network feedforward control is used for the tracking control of manipulators. The advantages of the compound control are that the system model does not need to be identified beforehand in the manipulator control system and it can achieve better adaptive control in an on-line continuous learning manner. The simulation results for the two-link manipulator show that the proposed compound control has higher tracking accuracy and better robustness than the conventional PD controllers in the position trajectory tracking control for the manipulator. Therefore, the compound cosine function neural network controller provides a novel approach for the manipulator control with uncertain nonlinear problems.     

Neural Network Identification Based Multivariable Feedback Linearization Robust Control for a Two-Link Manipulator

Regarding to the variations of the load and unmodeled dynamic, robot manipulators are known as a nonlinear dynamic system. Overcoming such problems like uncertainties and nonlinear characteristics in the model of two-link manipulator is the principal goal of this paper. To approach to this aim, a neural network is combined with a linear robust control in which the result has the advantages of, the first, approximated nonlinear elements and the second, the guaranteed robustness. To design the proposed controller, at first, multivariable feedback linearization is employed to convert the nonlinear model to linear one. Second, the unknown parameters of the system are identified by neural network based on a new proposed learning rule. Third, Mixed linear feedback-H_?∞? robust control method is proposed to stabilize the closed loop system. The closed loop system based on the proposed controller is analyzed and some numerical simulations are performed. Results show suitable responses of the closed loop system.     

Robot learning control based on recurrent neural network inverse model

A non-linear model-based feedforward, feedback, and learning controller is presented. This controller can control a non-linear plant such as a robot whose dynamics are initially unknown. In the feedforward part, a recurrent neural network (RNN) is used to model the inverse dynamics of the plant. In the feedback part, a PD controller is added to handle unmodeled dynamics and disturbances. Furthermore, an add-on learning controller is established to reduce tracking errors for repetitive tasks. The controller is validated with the control of a simulated two-joint manipulator. Simulation results show that the controller can successfully learn the inverse dynamics of a robot, perform accurate tracking for a general trajectory, and improve its own performance over the repetitions of a trajectory, with and without a payload change. © 1997 John Wiley & Sons, Inc.     

A Neural Net Predictive Control for Telerobots with Time Delay

This paper extends the Smith Predictor feedback control structure to unknown robotic systems in a rigorous fashion. A new recurrent neural net predictive control (RNNPC) strategy is proposed to deal with input and feedback time delays in telerobotic systems. The proposed control structure consists of a local linearized subsystem and a remote predictive controller. In the local linearized subsystem, a recurrent neural network (RNN) with on-line weight tuning algorithm is employed to approximate the dynamics of the time-delay-free nonlinear plant. The remote controller is a modified Smith predictor for the local linearized subsystem which provides prediction and maintains the desirable tracking performance. Stability analysis is given in the sense of Lyapunov. The result is an adaptive compensation scheme for unknown telerobotic systems with time delays, uncertainties, and external disturbances. A simulation of a two-link robotic manipulator is provided to illustrate the effectiveness of the proposed control strategy.     

A robust adaptive tracking controller using neural networks for a class of nonlinear systems

A neural networkbased robust adaptive tracking control scheme is proposed for a class of nonlinear systems. It is shown that, unlike most neural control schemes using the back-propagation training technique, one hidden layer neural controller is designed in the Lyapunov sense, and the parameters of the neural controller are then adaptively adjusted for the compensation of unknown dynamics and nonlinearities. Using this scheme, not only strong robustness with respect to unknown dynamics and nonlinearities can be obtained, but also asymptotic error convergence between the plant output and the reference model output can be guaranteed. A simulation example based on a one-link non-linear robotic manipulator is given in support of the proposed neural control scheme.     

Nonlinear asymptotic attitude tracking control of an underactuated 3-degree-of-freedom helicopter using neural network feedforward term

This paper proposes a new asymptotic attitude tracking controller for an underactuated 3-degree-of-freedom (DOF) laboratory helicopter system by using a nonlinear robust feedback and a neural network (NN) feedforward term. The nonlinear robust control law is developed through a modified inner-outer loop approach. The application of the NN-based feedforward is to compensate for the system uncertainties. The proposed control design strategy requires very limited knowledge of the system dynamic model, and achieves good robustness with respect to system parametric uncertainties. A Lyapunov-based stability analysis shows that the proposed algorithms can ensure asymptotic tracking of the helicopter’s elevation and travel motion, while keeping the stability of the closed-loop system. Real-time experiment results demonstrate that the controller has achieved good tracking performance.     

Neural-network-based robust adaptive control for a class of nonlinear systems

In this study, a robust adaptive control (RAC) system is developed for a class of nonlinear systems. The RAC system is comprised of a computation controller and a robust compensator. The computation controller containing a radial basis function (RBF) neural network is the principal controller, and the robust compensator can provide the smooth and chattering-free stability compensation. The RBF neural network is used to approximate the system dynamics, and the adaptive laws are derived to on-line tune the parameters of the neural network so as to achieve favorable estimation performance. From the Lyapunov stability analysis, it is shown that all signals in the closed-loop RBAC system are uniformly ultimately bounded. To investigate the effectiveness of the RAC system, the design methodology is applied to control two nonlinear systems: a wing rock motion system and a Chua’s chaotic circuit system. Simulation results demonstrate that the proposed RAC system can achieve favorable tracking performance with unknown of the system dynamics.     

基于神经网络的小型无人直升机非线性鲁棒控制设计

针对小型无人直升机的姿态控制问题,为补偿系统参数不确定性和外界扰动的影响,设计一种连续的非线性鲁棒控制器.首先,利用神经网络在线估计系统不确定性,采用基于误差符号函数积分的鲁棒控制算法抑制外界扰动,同时补偿神经网络估计误差; 然后,利用基于Lyapunov函数的分析方法,证明所设计控制器的闭环稳定性,确保无人直升机姿态误差的半全局渐近收敛;最后,在无人直升机飞行实验平台上,进行无人机抗风扰控制实验.实验结果表明,所提出的控制方法具有良好的控制效果,对系统不确定性和外界扰动具有良好的鲁棒性.     

时滞不确定飞行系统神经网络非脆弱H∞控制

针对具有参数摄动和状态时延的时滞不确定飞行系统,提出了一种神经网络非脆弱H_∞控制方案。该方案将鲁棒H_∞控制和神经网络控制结合起来,利用径向基神经网络的非线性逼近能力,对飞行系统的非线性不确定项进行逼近。由线性矩阵不等式（LMI）设计系统标称部分的鲁棒控制器,然后利用神经网络的输出来消除系统控制输入中的不确定部分。Lyapunov稳定性分析中,综合考虑了系统参数摄动、时延和神经网络逼近误差的影响,并证明了在所设计的飞行控制器作用下,闭环系统的稳定性。仿真实例验证了提出的飞行控制方案的可行性和有效性。     

Adaptive neural control for a class of uncertain nonlinear systems with unknown time delay

A neural network (NN)‐based robust adaptive control design scheme is developed for a class of nonlinear systems represented by input–output models with an unknown nonlinear function and unknown time delay. By approximating on‐line the unknown nonlinear functions with a three‐layer feedforward NN, the proposed approach does not require the unknown parameters to satisfy the linear dependence condition. The control law is delay independent and possible controller singularity problem is avoided. It is proved that with the proposed neural control law, all the signals in the closed‐loop system are semiglobally bounded in the presence of unknown time delay and unknown nonlinearity. A simulation example is presented to demonstrate the method. Copyright © 2008 John Wiley & Sons, Ltd.     

采样非线性系统的神经网络稳定自适应控制

针对一类动力学未知或难以建模的采样非线性系统,提出了一种基于神经网络的跟随控 制器稳定自适应控制方法.控制器采用径向基函数神经网络近似对象的动力学非线性,神经 网络参数的自适应规律由稳定理论得到.文中给出了系统稳定性和跟随误差收敛性的证明, 并通过仿真实例揭示了所提方法的性能.     

Time‐varying input delay compensation for nonlinear systems with additive disturbance: An output feedback approach

This paper addresses the output feedback tracking control of a class of multiple‐input and multiple‐output nonlinear systems subject to time‐varying input delay and additive bounded disturbances. Based on the backstepping design approach, an output feedback robust controller is proposed by integrating an extended state observer and a novel robust controller, which uses a desired trajectory‐based feedforward term to achieve an improved model compensation and a robust delay compensation feedback term based on the finite integral of the past control values to compensate for the time‐varying input delay. The extended state observer can simultaneously estimate the unmeasurable system states and the additive disturbances only with the output measurement and delayed control input. The proposed controller theoretically guarantees prescribed transient performance and steady‐state tracking accuracy in spite of the presence of time‐varying input delay and additive bounded disturbances based on Lyapunov stability analysis by using a Lyapunov‐Krasovskii functional. A specific study on a 2‐link robot manipulator is performed; based on the system model and the proposed design procedure, a suitable controller is developed, and comparative simulation results are obtained to demonstrate the effectiveness of the developed control scheme.     

Synchronization of general chaotic systems using neural controllers with application to secure communication

The main contribution of this paper is to propose a nonlinear robust controller to synchronize general chaotic systems, such that the controller does not need the information of the chaotic system’s model. Following this purpose, in this paper, two methods are proposed to synchronize general forms of chaotic systems with application in secure communication. The first method uses radial basis function neural network (RBFNN) as a controller. All the parameters of the RBFNN are derived and optimized via particle swarm optimization (PSO) algorithm and genetic algorithm (GA). In order to increase the robustness of the controller, in the second method, an integral term is added to the RBF neural network gives an integral RBFNN (IRBFNN). The coefficients of the integral term and the parameters of IRBFNN are also derived and optimized via PSO and GA. The proposed methods are applied to the famous Lorenz chaotic system for secure communication. The performance and control effort of the proposed methods are compared with the recently proposed PID controller optimized via GA. Simulation results show the superiority of the proposed methods in comparison to the recent one in improving synchronization while using smaller control effort.     

A new RISE-based adaptive control of PKMs: design,stability analysis and experiments

This paper deals with the development of a new adaptive control scheme for parallel kinematic manipulators (PKMs) based on Rrbust integral of the sign of the error (RISE) control theory. Original RISE control law is only based on state feedback and does not take advantage of the modelled dynamics of the manipulator. Consequently, the overall performance of the resulting closed-loop system may be poor compared to modern advanced model-based control strategies. We propose in this work to extend RISE by including the nonlinear dynamics of the PKM in the control loop to improve its overall performance. More precisely, we augment original RISE control scheme with a model-based adaptive control term to account for the inherent nonlinearities in the closed-loop system. To demonstrate the relevance of the proposed controller, real-time experiments are conducted on the Delta robot, a three-degree-of-freedom (3-DOF) PKM.     

Neural network controllers for robot manipulators application of damping neurons

^_This paper presents an effective adaptive neural network feedback controller for force control of robot manipulators in an unknown environment by applying damping neurons which possess elastic-viscous properties. The unexpected overshooting and oscillation caused by the unknown and/or unmodeled dynamics of a robot manipulator and an environment can be decreased efficiently by the effect of the proposed damping neurons. Furthermore, a fuzzy controlled evaluation function is applied for the learning of the proposed neural network controller, so that the controller is able to adapt to the unknown environment more effectively. The effectiveness of the proposed neural network controller is evaluated by experiment with a 3 d.o.f. direct-drive planar robot manipulator.     

An Adaptive RBF Neural Network Control Method for a Class of Nonlinear Systems

This paper focuses on designing an adaptive radial basis function neural network (RBFNN) control method for a class of nonlinear systems with unknown parameters and bounded disturbances. The problems raised by the unknown functions and external disturbances in the nonlinear system are overcome by RBFNN, combined with the single parameter direct adaptive control method. The novel adaptive control method is designed to reduce the amount of computations effectively. The uniform ultimate boundedness of the closed-loop system is guaranteed by the proposed controller. A coupled motor drives (CMD) system, which satisfies the structure of nonlinear system, is taken for simulation to confirm the effectiveness of the method. Simulations show that the developed adaptive controller has favorable performance on tracking desired signal and verify the stability of the closed-loop system.      

Two-degree-of-freedom H-infinity control of combustion in diesel engine using a discrete dynamics model

This paper proposes an H-infinity combustion control method for diesel engines. The plant model is the discrete dynamics model developed by Yasuda et al., which is implementable on a real engine control unit. We introduce a two-degree-of-freedom control scheme with a feedback controller and a feedforward controller. This scheme achieves both good feedback properties, such as disturbance suppression and robust stability, and a good transient response. The feedforward controller is designed by taking the inverse of the static plant model, and the feedback controller is designed by the H-infinity control method, which reduces the effect of the trubocharger lag. The effectiveness of the proposed method is evaluated in simulations using the nonlinear discrete dynamics model.