Cooperative learning neural network output feedback control of uncertain nonlinear multi-agent systems under directed topologies

Without assuming that the communication topologies among the neural network (NN) weights are to be undirected and the states of each agent are measurable, the cooperative learning NN output feedback control is addressed for uncertain nonlinear multi-agent systems with identical structures in strict-feedback form. By establishing directed communication topologies among NN weights to share their learned knowledge, NNs with cooperative learning laws are employed to identify the uncertainties. By designing NN-based κ-filter observers to estimate the unmeasurable states, a new cooperative learning output feedback control scheme is proposed to guarantee that the system outputs can track nonidentical reference signals with bounded tracking errors. A simulation example is given to demonstrate the effectiveness of the theoretical results.     

Consensus-based distributed cooperative learning control for a group of discrete-time nonlinear multi-agent systems using neural networks

This paper focuses on the cooperative learning capability of radial basis function neural networks in adaptive neural controllers for a group of uncertain discrete-time nonlinear systems where system structures are identical but reference signals are different. By constructing an interconnection topology among learning laws of NN weights in order to share their learned knowledge on-line, a novel adaptive NN control scheme, called distributed cooperative learning control scheme, is proposed. It is guaranteed that if the interconnection topology is undirected and connected, all closed-loop signals are uniform ultimate bounded and tracking errors of all systems can converge to a small neighborhood around the origin. Moreover, it is proved that all estimated NN weights converge to a small neighborhood of their common optimal value along the union of all state trajectories, which means that the estimated NN weights reach consensus with a small consensus error. Thus, all learned NN models have the better generalization capability than ones obtained by the deterministic learning method. The learned knowledge is also adopted to control a class of uncertain systems with the same structure but different reference signals. Finally, a simulation example is provided to verify the effectiveness and advantages of the distributed cooperative learning control scheme developed in this paper.     

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

This paper introduces an observer-based adaptive optimal control method for unknown singularly perturbed nonlinear systems with input constraints. First, a multi-time scales dynamic neural network (MTSDNN) observer with a novel updating law derived from a properly designed Lyapunov function is proposed to estimate the system states. Then, an adaptive learning rule driven by the critic NN weight error is presented for the critic NN, which is used to approximate the optimal cost function. Finally, the optimal control action is calculated by online solving the Hamilton-Jacobi-Bellman (HJB) equation associated with the MTSDNN observer and critic NN. The stability of the overall closed-loop system consisting of the MTSDNN observer, the critic NN and the optimal control action is proved. The proposed observer-based optimal control approach has an essential advantage that the system dynamics are not needed for implementation, and only the measured input/output data is needed. Moreover, the proposed optimal control design takes the input constraints into consideration and thus can overcome the restriction of actuator saturation. Simulation results are presented to confirm the validity of the investigated approach.      

Adaptive neural control and learning of affine nonlinear systems

This paper presents deterministic learning from adaptive neural network control of affine nonlinear systems with completely unknown system dynamics. Thanks to the learning capability of radial basis function, neural network (NN), stable adaptive NN controller is designed for the unknown affine nonlinear systems. The designed adaptive NN controller is rigorously shown that learning of the unknown closed-loop system dynamics can be achieved during the stable control process because partial persistent excitation condition of some internal signals in the closed-loop system is satisfied. Subsequently, neural learning controller using the knowledge obtained from deterministic learning is constructed to achieve closed-loop stability and improve control performance. Numerical simulation is provided to show the effectiveness of the proposed control scheme.     

Reinforcement learning-based output feedback control of nonlinear systems with input constraints.

A novel neural network (NN)-based output feedback controller with magnitude constraints is designed to deliver a desired tracking performance for a class of multi-input and multi-output (MIMO) strict feedback nonlinear discrete-time systems. Reinforcement learning is proposed for the output feedback controller, which uses three NNs: 1) an NN observer to estimate the system states with the input-output data, 2) a critic NN to approximate certain strategic utility function, and 3) an action NN to minimize both the strategic utility function and the unknown dynamics estimation errors. Using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the state estimation errors, the tracking errors and weight estimates is shown.     

Distributed Cooperative Learning for Discrete-Time Strict-Feedback Multi Agent Systems Over Directed Graphs

This paper focuses on the distributed cooperative learning (DCL) problem for a class of discrete-time strict-feedback multi-agent systems under directed graphs. Compared with the previous DCL works based on undirected graphs, two main challenges lie in that the Laplacian matrix of directed graphs is nonsymmetric, and the derived weight error systems exist n-step delays. Two novel lemmas are developed in this paper to show the exponential convergence for two kinds of linear time-varying (LTV) systems with different phenomena including the nonsymmetric Laplacian matrix and time delays. Subsequently, an adaptive neural network (NN) control scheme is proposed by establishing a directed communication graph along with n-step delays weight updating law. Then, by using two novel lemmas on the extended exponential convergence of LTV systems, estimated NN weights of all agents are verified to exponentially converge to small neighbourhoods of their common optimal values if directed communication graphs are strongly connected and balanced. The stored NN weights are reused to structure learning controllers for the improved control performance of similar control tasks by the “mod” function and proper time series. A simulation comparison is shown to demonstrate the validity of the proposed DCL method.      

Learning from neural control

One of the amazing successes of biological systems is their ability to "learn by doing" and so adapt to their environment. In this paper, first, a deterministic learning mechanism is presented, by which an appropriately designed adaptive neural controller is capable of learning closed-loop system dynamics during tracking control to a periodic reference orbit. Among various neural network (NN) architectures, the localized radial basis function (RBF) network is employed. A property of persistence of excitation (PE) for RBF networks is established, and a partial PE condition of closed-loop signals, i.e., the PE condition of a regression subvector constructed out of the RBFs along a periodic state trajectory, is proven to be satisfied. Accurate NN approximation for closed-loop system dynamics is achieved in a local region along the periodic state trajectory, and a learning ability is implemented during a closed-loop feedback control process. Second, based on the deterministic learning mechanism, a neural learning control scheme is proposed which can effectively recall and reuse the learned knowledge to achieve closed-loop stability and improved control performance. The significance of this paper is that the presented deterministic learning mechanism and the neural learning control scheme provide elementary components toward the development of a biologically-plausible learning and control methodology. Simulation studies are included to demonstrate the effectiveness of the approach.     

Near Optimal Output Feedback Control of Nonlinear Discrete-time Systems Based on Reinforcement Neural Network Learning

In this paper, the output feedback based finitehorizon near optimal regulation of nonlinear affine discretetime systems with unknown system dynamics is considered by using neural networks (NNs) to approximate Hamilton-Jacobi-Bellman (HJB) equation solution. First, a NN-based Luenberger observer is proposed to reconstruct both the system states and the control coefficient matrix. Next, reinforcement learning methodology with actor-critic structure is utilized to approximate the time-varying solution, referred to as the value function, of the HJB equation by using a NN. To properly satisfy the terminal constraint, a new error term is defined and incorporated in the NN update law so that the terminal constraint error is also minimized over time. The NN with constant weights and timedependent activation function is employed to approximate the time-varying value function which is subsequently utilized to generate the finite-horizon near optimal control policy due to NN reconstruction errors. The proposed scheme functions in a forward-in-time manner without offline training phase. Lyapunov analysis is used to investigate the stability of the overall closedloop system. Simulation results are given to show the effectiveness and feasibility of the proposed method.      

Adaptive critic learning for event-triggered safe control of nonlinear safety-critical systems

In this paper, an event-triggered safe control method based on adaptive critic learning (ACL) is proposed for a class of nonlinear safety-critical systems. First, a safe cost function is constructed by adding a control barrier function (CBF) to the traditional quadratic cost function; the optimization problem with safety constraints that is difficult to deal with by classical ACL methods is solved. Subsequently, the event-triggered scheme is introduced to reduce the amount of computation. Further, combining the properties of CBF with the ACL-based event-triggering mechanism, the event-triggered safe Hamilton–Jacobi–Bellman (HJB) equation is derived, and a single critic neural network (NN) framework is constructed to approximate the solution of the event-triggered safe HJB equation. In addition, the concurrent learning method is applied to the NN learning process, so that the persistence of excitation (PE) condition is not required. The weight approximation error of the NN and the states of the system are proven to be uniformly ultimately bounded (UUB) in the safe set with the Lyapunov theory. Finally, the availability of the presented method can be validated through the simulation.     

Output-Feedback Based Simplified Optimized Backstepping Control for Strict-Feedback Systems with Input and State Constraints

In this paper, an adaptive neural-network (NN) output feedback optimal control problem is studied for a class of strict-feedback nonlinear systems with unknown internal dynamics, input saturation and state constraints. Neural networks are used to approximate unknown internal dynamics and an adaptive NN state observer is developed to estimate immeasurable states. Under the framework of the backstepping design, by employing the actor-critic architecture and constructing the tan-type Barrier Lyapunov function (BLF), the virtual and actual optimal controllers are developed. In order to accomplish optimal control effectively, a simplified reinforcement learning (RL) algorithm is designed by deriving the updating laws from the negative gradient of a simple positive function, instead of employing existing optimal control methods. In addition, to ensure that all the signals in the closed-loop system are bounded and the output can follow the reference signal within a bounded error, all state variables are confined within their compact sets all times. Finally, a simulation example is given to illustrate the effectiveness of the proposed control strategy.      

Projection-Based Adaptive Neurocontrol With Switching Logic Deadzone Tuning

In this brief, an adaptive neural network (NN) controller is proposed for multiple-input-multiple-output (MIMO) nonlinear systems with triangular control structure and unknown control directions. Deadzones are employed in the projection-based NN weight learning laws and the Nussbaum parameter update laws with levels tuned by an innovative switching logic tuning mechanism. Detailed analysis using a family of Lyapunov-like integral functions and the function approximation capability of NNs proves that all the tracking errors are semiglobal uniform ultimate bounded in small neighborhoods of the origin while the closed-loop system variables (state vector, NN weights, Nussbaum parameters) and the control law remain bounded. A simulation study confirms the theoretical results and verifies the effectiveness of the proposed design.     

基于神经网络模型的直接优化预测控制

针对具有时延的非线性系统提出了一种基于神经网络模型直接优于的预测控制。     

Adaptive neural output feedback control for stochastic nonlinear time-delay systems with unknown control directions

This paper first focuses on the problem of adaptive output feedback stabilization for a more general class of stochastic nonlinear time-delay systems with unknown control directions. By using a linear state transformation, the original system is transformed to a new system for which control design becomes feasible. Then a novel adaptive neural network (NN) output feedback control strategy, which only contains one adaptive parameter, is developed for such systems by combining the input-driven filter design, the backstepping technique, the NN’s parameterization, the Nussbaum gain function method and the Lyapunov–Krasovskii approach. The proposed control design guarantees that all signals in the closed-loop systems are 4-moment (or 2-moment) semi-globally uniformly bounded. Finally, two simulation examples are given to demonstrate the effectiveness and the applicability of the proposed control design.     

Neural Network Adaptive Robust Control Of Siso Nonlinear Systems In A Normal Form

In this paper, performance oriented control laws are synthesized for a class of single‐input‐single‐output (SISO) n‐th order nonlinear systems in a normal form by integrating the neural networks (NNs) techniques and the adaptive robust control (ARC) design philosophy. All unknown but repeat‐able nonlinear functions in the system are approximated by the outputs of NNs to achieve a better model compensation for an improved performance. While all NN weights are tuned on‐line, discontinuous projections with fictitious bounds are used in the tuning law to achieve a controlled learning. Robust control terms are then constructed to attenuate model uncertainties for a guaranteed output tracking transient performance and a guaranteed final tracking accuracy. Furthermore, if the unknown nonlinear functions are in the functional ranges of the NNs and the ideal NN weights fall within the fictitious bounds, asymptotic output tracking is achieved to retain the perfect learning capability of NNs. The precision motion control of a linear motor drive system is used as a case study to illustrate the proposed NNARC strategy.     

Event-triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs

In this article, the event-triggered optimal tracking control problem for multiplayer unknown nonlinear systems is investigated by using adaptive critic designs. By constructing a neural network (NN)-based observer with input–output data, the system dynamics of multiplayer unknown nonlinear systems is obtained. Subsequently, the optimal tracking control problem is converted to an optimal regulation problem by establishing a tracking error system. Then, the optimal tracking control policy for each player is derived by solving coupled event-triggered Hamilton-Jacobi (HJ) equation via a critic NN. Meanwhile, a novel weight updating rule is designed by adopting concurrent learning method to relax the persistence of excitation (PE) condition. Moreover, an event-triggering condition is designed by using Lyapunov's direct method to guarantee the uniform ultimate boundedness (UUB) of the closed-loop multiplayer systems. Finally, the effectiveness of the developed method is verified by two different multiplayer nonlinear systems.     

Deterministic learning from neural control for uncertain nonlinear pure‐feedback systems by output feedback

The essence of intelligence lies in the acquisition/learning and utilization of knowledge. However, how to implement learning in dynamical environments for nonlinear systems is a challenging issue. This article investigates the deterministic learning (DL) control problem for uncertain pure‐feedback systems by output feedback, which achieves the human‐like learning and control in a simple way. To reduce the complexity of control design and analysis, first, by combining an appropriate system transformation, the original pure‐feedback system is transformed into a simple normal nonaffine system. An observer is then introduced to estimate the transformed system states. Based on the backstepping and dynamic surface control techniques, a simple adaptive neural control scheme is first developed to guarantee the finite time convergence of the tracking error using only one neural network (NN) approximator. Second, through DL, the exponential convergence of the NN weights is obtained with the satisfaction of partial persistent excitation condition. Thus, locally accurate approximation/learning of the transformed unknown system dynamics is achieved and stored as constant NNs. Finally, by utilizing the stored knowledge, an experience‐based controller is constructed and a novel learning control scheme is further proposed to improve the control performance without any further adaptation online for the estimate neural weights. Simulation results have been given to illustrate that the proposed scheme not only can learn and memorize knowledge like humans but also can utilize experience to achieve superior control performance.     

Distributed cooperative reinforcement learning for multi-agent system with collision avoidance

In this work, we present an optimal cooperative control scheme for a multi-agent system in an unknown dynamic obstacle environment, based on an improved distributed cooperative reinforcement learning (RL) strategy with a three-layer collaborative mechanism. The three collaborative layers are collaborative perception layer, collaborative control layer, and collaborative evaluation layer. The incorporation of collaborative perception expands the perception range of a single agent, and improves the early warning ability of the agents for the obstacles. Neural networks (NNs) are employed to approximate the cost function and the optimal controller of each agent, where the NN weight matrices are collaboratively optimized to achieve global optimal performance. The distinction of the proposed control strategy is that cooperation of the agents is embodied not only in the input of NNs (in a collaborative perception layer) but also in their weight updating procedure (in the collaborative evaluation and collaborative control layers). Comparative simulations are carried out to demonstrate the effectiveness and performance of the proposed RL-based cooperative control scheme.     

采样非线性系统的动态神经网络稳定自适应控制

研究了一类采样数据非线性系统的动态神经网络稳定自适应控制方法.不同于静态神经网络自适应控制,动态神经网络自适应控制中神经网络用于逼近整个采样数据非线性系统,而不是动态系统中的非线性分量.系统的控制律由神经网络系统的动态逆、自适应补偿项和神经变结构鲁棒控制项组成.神经变结构控制用于保证系统的全局稳定性,并加速动态神经网络系统的适近速度.证明了动态神经网络自适应控制系统的稳定性,并得到了动态神经网络系统的学习算法.仿真研究表明,基于动态神经网络的非线性系统稳定自适应控制方法较基于静态神经网络的自适应方法具有更好的性能.     

Discrete-time neural net controller for a class of nonlineardynamical systems

A family of two-layer discrete-time neural net (NN) controllers is presented for the control of a class of mnth-order MIMO dynamical system. No initial learning phase is needed so that the control action is immediate; in other words, the neural network (NN) controller exhibits a learning-while-functioning-feature instead of a learning-then-control feature. A two-layer NN is used which is linear in the tunable weights. The structure of the neural net controller is derived using a filtered error approach. It is indicated that delta-rule-based tuning, when employed for closed-loop control, can yield unbounded NN weights if: 1) the net cannot exactly reconstruct a certain required function, or 2) there are bounded unknown disturbances acting on the dynamical system. Certainty equivalence is not used, overcoming a major problem in discrete-time adaptive control. In this paper, new online tuning algorithms for discrete-time systems are derived which are similar to ϵ-modification for the case of continuous-time systems that include a modification to the learning rate parameter and a correction term to the standard delta rule     

Discrete time neural network synthesis using input and outputactivation functions

A new very fast algorithm for synthesis of a new structure of discrete-time neural networks (NN) is proposed. For this purpose the following concepts are employed: (i) combination of input and output activation functions, (ii) input time-varying signal distribution, (iii) time-discrete domain synthesis and (iv) one-step learning iteration approach. The problem of input-output mappings of time-varying vectors is solved. Simulation results based on the synthesis of a new structure of feedforward NN of an universal logical unit are presented. The proposed NN synthesis procedure is useful for applications to identification and control of nonlinear, very fast, dynamical systems. In this sense a feedforward NN for an adaptive nonlinear robot control is designed. Finally, a new algorithm for the direct inverse modeling of input/output nonquadratic systems is discussed.