An adaptive critic approach to event‐triggered robust control of nonlinear systems with unmatched uncertainties

In this paper, we develop a novel event‐triggered robust control strategy for continuous‐time nonlinear systems with unmatched uncertainties. First, we build a relationship to show that the event‐triggered robust control can be obtained by solving an event‐triggered nonlinear optimal control problem of the auxiliary system. Then, within the framework of reinforcement learning, we propose an adaptive critic approach to solve the event‐triggered nonlinear optimal control problem. Unlike typical actor‐critic dual approximators used in reinforcement learning, we employ a unique critic approximator to derive the solution of the event‐triggered Hamilton‐Jacobi‐Bellman equation arising in the nonlinear optimal control problem. The critic approximator is updated via the gradient descent method, and the persistence of excitation condition is necessary. Meanwhile, under a newly proposed event‐triggering condition, we prove that the developed critic approximator update rule guarantees all signals in the auxiliary closed‐loop system to be uniformly ultimately bounded. Moreover, we demonstrate that the obtained event‐triggered optimal control can ensure the original system to be stable in the sense of uniform ultimate boundedness. Finally, a F‐16 aircraft plant and a nonlinear system are provided to validate the present event‐triggered robust control scheme.     

NNs‐Based Event‐Triggered Consensus Control of a Class of Uncertain Nonlinear Multi‐Agent Systems

We are concerned with the consensus problem for a class of uncertain nonlinear multi‐agent systems (MASs) connected through an undirected communication topology via event‐triggered approaches in this paper. Two distributed control strategies, the adaptive centralized event‐triggered control one and adaptive distributed event‐triggered control one, are presented utilizing neural networks (NNs) and event‐driven mechanisms, where the advantages of the proposed control laws lie that they remove the requirement for exact priori knowledge about parameters of individual agents by taking advantage of NNs approximators and they save computing and communication resources since control tasks only execute at certain instants with respect to predefined threshold functions. Also, the trigger coefficient can be regulated adaptively with dependence on state errors to ensure not only the control performance but also the efficiency of the network interactions. It is proven that all signals in the closed‐loop system are bounded and the Zeno behavior is excluded. Finally, simulation examples are presented for illustration of the theoretical claims.     

Fixed‐time sliding mode cooperative control for multiagent networks via event‐triggered strategy

In this article, the problem of event‐triggered‐based fixed‐time sliding mode cooperative control is addressed for a class of leader‐follower multiagent networks with bounded perturbation. First, a terminal integral sliding mode manifold with fast convergent speed is designed. Then, a distributed consensus tracking control strategy based on event‐triggered and sliding mode control is developed that guarantees the multiagent networks achieve consensus within a fixed time which is independent of initial states of agents in comparison with the finite‐time convergence. Furthermore, the update frequency of control law can be considerably reduced and Zeno behavior can be removed by utilizing the proposed event‐triggered control algorithm. Simulation examples are used to show the effectiveness of the new control protocol.     

不对称约束多人非零和博弈的自适应评判控制

本文针对连续时间非线性系统的不对称约束多人非零和博弈问题, 建立了一种基于神经网络的自适应评判控制方法. 首先, 本文提出了一种新颖的非二次型函数来处理不对称约束问题, 并且推导出最优控制律和耦合Hamilton-Jacobi方程. 值得注意的是, 当系统状态为零时, 最优控制策略是不为零的, 这与以往不同. 然后, 通过构建单一评判网络来近似每个玩家的最优代价函数, 从而获得相关的近似最优控制策略. 同时, 在评判学习期间发展了一种新的权值更新规则. 此外, 通过利用Lyapunov理论证明了评判网络权值近似误差和闭环系统状态的稳定性. 最后, 仿真结果验证了本文所提方法的有效性     

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

This paper introduces an observer-based adaptive optimal control method for unknown singularly perturbed nonlinear systems with input constraints. First, a multi-time scales dynamic neural network (MTSDNN) observer with a novel updating law derived from a properly designed Lyapunov function is proposed to estimate the system states. Then, an adaptive learning rule driven by the critic NN weight error is presented for the critic NN, which is used to approximate the optimal cost function. Finally, the optimal control action is calculated by online solving the Hamilton-Jacobi-Bellman (HJB) equation associated with the MTSDNN observer and critic NN. The stability of the overall closed-loop system consisting of the MTSDNN observer, the critic NN and the optimal control action is proved. The proposed observer-based optimal control approach has an essential advantage that the system dynamics are not needed for implementation, and only the measured input/output data is needed. Moreover, the proposed optimal control design takes the input constraints into consideration and thus can overcome the restriction of actuator saturation. Simulation results are presented to confirm the validity of the investigated approach.      

Neural-based online finite-time optimal tracking control for wheeled mobile robotic system with inequality constraints

In this study, a finite-time online optimal controller was designed for a nonlinear wheeled mobile robotic system (WMRS) with inequality constraints, based on reinforcement learning (RL) neural networks. In addition, an extended cost function, obtained by introducing a penalty function to the original long-time cost function, was proposed to deal with the optimal control problem of the system with inequality constraints. A novel Hamilton-Jacobi-Bellman (HJB) equation containing the constraint conditions was defined to determine the optimal control input. Furthermore, two neural networks (NNs), a critic and an actor NN, were established to approximate the extended cost function and the optimal control input, respectively. The adaptation laws of the critic and actor NN were obtained with the gradient descent method. The semi-global practical finite-time stability (SGPFS) was proved using Lyapunov's stability theory. The tracking error converges to a small region near zero within the constraints in a finite period. Finally, the effectiveness of the proposed optimal controller was verified by a simulation based on a practical wheeled mobile robot model.     

Fuzzy adaptive dynamic programming-based optimal leader-following consensus for heterogeneous nonlinear multi-agent systems

In this paper, a novel online iterative scheme, based on fuzzy adaptive dynamic programming, is proposed for distributed optimal leader-following consensus of heterogeneous nonlinear multi-agent systems under directed communication graph. This scheme combines game theory, adaptive dynamic programming together with generalized fuzzy hyperbolic model (GFHM). Firstly, based on precompensation technique, an appropriate model transformation is proposed to convert the error system into augmented error system, and an exquisite performance index function is defined for this system. Secondly, on the basis of Hamilton–Jacobi–Bellman (HJB) equation, the optimal consensus control is designed and a novel policy iteration (PI) algorithm is put forward to learn the solutions of the HJB equation online. Here, the proposed PI algorithm is implemented on account of GFHMs. Compared with dual-network model including critic network and action network, the proposed scheme only requires critic network. Thirdly, the augmented consensus error of each agent and the weight estimation error of each GFHM are proved to be uniformly ultimately bounded, and the stability of our method has been verified. Finally, some numerical examples and application examples are conducted to demonstrate the effectiveness of the theoretical results.

     

Distributed event‐triggered containment control for dynamical multiagent networks

This paper studies the event‐triggered containment control problem for dynamical multiagent networks of general MIMO linear agents. An event‐triggered containment control strategy is provided, which consists of a control law based on a relative‐state feedback and a distributed triggering rule based on both the relative‐state information and a time‐dependent threshold function. Compared to the previous related works, our main contribution is that the triggering rule depends only on local information of communication networks. It is proved that under the proposed event‐based controller, the containment errors are uniformly ultimately bounded and the Zeno behavior can be excluded. Moreover, when the derivation constant in the threshold function is equal to zero, the containment control problem can be solved. Then, the results are extended to the event‐triggered observer‐based containment controller design.     

Node‐to‐node consensus of multi‐agent networks with event‐triggered control and packet losses

This paper studies the node‐to‐node consensus problem for multi‐agent networks possessing a leaders' layer and a followers' layer via the pinning control. In order to realize the consensus and reduce the update frequency of the controller, a suitable event‐triggered mechanism is introduced into the control strategy. Furthermore, the phenomenon of packet loss is considered in the designed controller. Based on the M‐matrix theory and Lyapunov stability theory, this paper presents the sufficient conditions for the node‐to‐node consensus of networks. Meanwhile, it is proved that the Zeno behaviour is excluded. Finally, two numerical simulations are provided to demonstrate the effectiveness of the theoretical results.     

Adaptive event‐triggered neural control for nonlinear uncertain system with input constraint

In this article, the issue of developing an adaptive event‐triggered neural control for nonlinear uncertain system with input delay is investigated. The radial basis function neural networks (RBFNNs) are adopted to approximate the uncertain terms, where the time‐varying approximation errors are considered into the approximation system. However, the RBFNNs' weight vector is extended, which may cause the computing burdens. To save network resource, the computing burden caused by the weight vector is handled with the developed adaptive control strategy. Furthermore, in order to compensate the effect of input delay, an auxiliary system is introduced into codesign. With the help of adaptive backstepping technique, an adaptive event‐triggered control approach is established. Under the proposed control approach, the effect of input delay can be compensated effectively while the considered system suffered network resource constraint, and all signals in the close‐loop system can be guarantee bounded. Finally, two simulation examples are given to verify the proposed control method's effectiveness.     

Observer‐based leaderless and leader‐following consensus for multiagent systems: A modified integral‐type event‐triggered design

This article aims to solve leaderless and leader‐following consensus problems for general linear systems by integral‐type event‐triggered control method. Different from the existing integral‐type event‐triggered controllers for multiagent systems (MASs), a modified distributed integral‐type event‐triggered scheme is designed via defining a measurement error without continuous communication. Then, distributed event‐triggered protocols are proposed for MASs to achieve the leaderless and leader‐following consensus. Moreover, for the case that all the agents' states are not available, distributed observers are given to estimate the full states. Meanwhile, leaderless and leader‐following consensus problems are investigated based on the observer‐based event‐triggered schemes. In addition, no agent will exhibit Zeno behavior. Finally, simulations are given to verify the effectiveness of our results.     

Robust event‐triggered model predictive control for constrained linear continuous system

Model predictive control (MPC) is capable to deal with multiconstraint systems in real control processes; however, the heavy computation makes it difficult to implement. In this paper, a dual‐mode control strategy based on event‐triggered MPC (ETMPC) and state‐feedback control for continuous linear time‐invariant systems including control input constraints and bounded disturbances is developed. First, the deviation between the actual state trajectory and the optimal state trajectory is computed to set an event‐triggered mechanism and reduce the computational load of MPC. Next, the dual‐mode control strategy is designed to stabilize the system. Both recursive feasibility and stability of the strategy are guaranteed by constructing a feasible control sequence and deducing the relationship of parameters, especially the inter‐event time and the upper bound of the disturbances. Finally, the theoretical results are supported by numerical simulation. In addition, the effects of the parameters are discussed by simulation, which gives guidance to balance computational load and control performance.     

A model-based deep reinforcement learning method applied to finite-horizon optimal control of nonlinear control-affine system

The Hamilton–Jacobi–Bellman (HJB) equation can be solved to obtain optimal closed-loop control policies for general nonlinear systems. As it is seldom possible to solve the HJB equation exactly for nonlinear systems, either analytically or numerically, methods to build approximate solutions through simulation based learning have been studied in various names like neurodynamic programming (NDP) and approximate dynamic programming (ADP). The aspect of learning connects these methods to reinforcement learning (RL), which also tries to learn optimal decision policies through trial-and-error based learning. This study develops a model-based RL method, which iteratively learns the solution to the HJB and its associated equations. We focus particularly on the control-affine system with a quadratic objective function and the finite horizon optimal control (FHOC) problem with time-varying reference trajectories. The HJB solutions for such systems involve time-varying value, costate, and policy functions subject to boundary conditions. To represent the time-varying HJB solution in high-dimensional state space in a general and efficient way, deep neural networks (DNNs) are employed. It is shown that the use of DNNs, compared to shallow neural networks (SNNs), can significantly improve the performance of a learned policy in the presence of uncertain initial state and state noise. Examples involving a batch chemical reactor and a one-dimensional diffusion-convection-reaction system are used to demonstrate this and other key aspects of the method.     

A neural composite dynamic surface control for pure‐feedback systems with unknown control gain signs and full state constraints

This paper investigates a composite neural dynamic surface control (DSC) method for a class of pure‐feedback nonlinear systems in the case of unknown control gain signs and full‐state constraints. Neural networks are utilized to approximate the compound unknown functions, and the approximation errors of neural networks are applied in the design of updated adaptation laws. Comparing the proposed composite approximation method with the conventional ones, a faster and better approximation performance result can be obtained. Combining the composite neural networks approximation with the DSC technique, an improved composite neural adaptive control approach is designed for the considered nonlinear system. Then, together with the Lyapunov stability theory, all the variables of the closed‐loop system are semiglobal uniformly ultimately bounded. The infringements of full state constraints can be avoided in the case of unknown control gain signs as well as unknown disturbances. Finally, two simulation examples show the effectiveness and feasibility of the proposed results.     

Distributed event‐triggered feedback consensus control with state‐dependent threshold for general linear multi‐agent systems

This paper considers the distributed event‐triggered consensus problem for multi‐agent systems with general linear dynamics under undirected graphs. Based on state feedback, we propose a novel distributed event‐triggered consensus controller with state‐dependent threshold for each agent to achieve consensus, without continuous communication in either controller update or triggering condition monitoring. Each agent only needs to monitor its own state continuously to determine if the event is triggered. It is proved that there is no Zeno behavior under the proposed consensus control algorithm. To relax the requirement of the state measurement of each agent, we further propose a novel distributed observer‐based event‐triggered consensus controller to solve the consensus problem in the case with output feedback and prove that there is no Zeno behavior exhibited. Finally, simulation results are given to illustrate the theoretical results. Copyright © 2016 John Wiley & Sons, Ltd.     

Adaptive finite‐time neural backstepping control for multi‐input and multi‐output state‐constrained nonlinear systems using tangent‐type nonlinear mapping

This article focuses on the problem of adaptive finite‐time neural backstepping control for multi‐input and multi‐output nonlinear systems with time‐varying full‐state constraints and uncertainties. A tan‐type nonlinear mapping function is first proposed to convert the strict‐feedback system into a new pure‐feedback one without constraints. Neural networks are utilized to cope with unknown functions. To improve learning performance, a composite adaptive law is designed using tracking error and approximate error. A finite‐time convergent differentiator is adopted to avoid the problem of “explosion of complexity.” By theoretical analysis, all the signals of system are proved to be bounded, the outputs can track the desired signals in a finite time, and full‐state constraints are not transgressed. Finally, comparative simulations are offered to confirm the validity of the proposed control scheme.     

Event‐triggered predictive control for networked control systems with network‐induced delays and packet dropouts

This paper presents an event‐triggered predictive control approach to stabilize a networked control system subject to network‐induced delays and packet dropouts, for which the states are not measurable. An observer‐based event generator is first designed according to the deviation between the state estimation at the current time and the one at the last trigger time. A predictive control scheme with a selector is then proposed to compensate the effect of network‐induced delays and packet dropouts. Sufficient conditions for stabilization of the networked control system are derived by solving linear matrix inequalities and the corresponding gains of the controller and the observer are obtained. It is shown that the event‐triggered implementation is able to realize reduction in communication and save bandwidth resources of feedback channel networks. A simulation example of an inverted pendulum model illustrates the efficacy of the proposed scheme.     

Adaptive neural control for multiagent systems with asymmetric time‐varying state constraints and input saturation

This article investigates the leader‐follower consensus problem of a class of non‐strict‐feedback nonlinear multiagent systems with asymmetric time‐varying state constraints (ATVSC) and input saturation, and an adaptive neural control scheme is developed. By introducing the distributed sliding‐mode estimator, each follower can obtain the estimation of leader's trajectory and track it directly. Then, with the help of time‐varying asymmetric barrier Lyapunov function and radial basis function neural networks, the controller is designed based on backstepping technique. Furthermore, the mean‐value theorem and Nussbaum function are utilized to address the problems of input saturation and unknown control direction. Moreover, the number of adaptive laws is equal to that of the followers, which reduces the computational complexity. It is proved that the leader‐follower consensus tracking control is achieved without violating the ATVSC, and all closed‐loop signals are semiglobally uniformly ultimately bounded. Finally, the simulation results are provided to verify the effectiveness of the control scheme.     

Distributed Adaptive Event‐Triggered Control for Leader‐Following Consensus of Multi‐Agent Systems

This paper studies the leader‐following consensus problem for Lipschitz nonlinear multi‐agent systems using novel event‐triggered controllers. A distributed adaptive law is introduced for the event‐based control strategy design such that the proposed controllers are independent of system parameters and only use the relative states of neighboring agents, and hence are fully distributed. Due to the introduction of an event‐triggered control scheme, the controller of the agent is only triggered at it's own event times, and thus reduces the amount of communication between controller and actuator and lowers the frequency of controller updates in practice. Based on a quadratic Lyapunov function, the event condition which uses only neighbor information and local computation at trigger instants is established. Infinite triggers within a finite time are also verified to be impossible. The effectiveness of the theoretical results are illustrated through simulation examples.     

Event‐triggered optimal tracking control of nonlinear systems

We propose a novel event‐triggered optimal tracking control algorithm for nonlinear systems with an infinite horizon discounted cost. The problem is formulated by appropriately augmenting the system and the reference dynamics and then using ideas from reinforcement learning to provide a solution. Namely, a critic network is used to estimate the optimal cost while an actor network is used to approximate the optimal event‐triggered controller. Because the actor network updates only when an event occurs, we shall use a zero‐order hold along with appropriate tuning laws to encounter for this behavior. Because we have dynamics that evolve in continuous and discrete time, we write the closed‐loop system as an impulsive model and prove asymptotic stability of the equilibrium point and Zeno behavior exclusion. Simulation results of a helicopter, a one‐link rigid robot under gravitation field, and a controlled Van‐der‐Pol oscillator are presented to show the efficacy of the proposed approach. Copyright © 2016 John Wiley & Sons, Ltd.