Distributed event-triggered optimal bipartite consensus control for multiagent systems with input delay via reinforcement learning method

In this article, an optimal bipartite consensus control (OBCC) scheme is proposed for heterogeneous multiagent systems (MASs) with input delay by reinforcement learning (RL) algorithm. A directed signed graph is established to construct MASs with competitive and cooperative relationships, and model reduction method is developed to tackle input delay problem. Then, based on the Hamilton–Jacobi–Bellman (HJB) equation, policy iteration method is utilized to design the bipartite consensus controller, which consists of value function and optimal controller. Further, a distributed event-triggered function is proposed to increase control efficiency, which only requires information from its own agent and neighboring agents. Based on the input-to-state stability (ISS) function and Lyapunov function, sufficient conditions for the stability of MASs can be derived. Apart from that, RL algorithm is employed to solve the event-triggered OBCC problem in MASs, where critic neural networks (NNs) and actor NNs estimate value function and control policy, respectively. Finally, simulation results are given to validate the feasibility and efficiency of the proposed algorithm.     

Event-triggered-based integral reinforcement learning output feedback optimal control for partially unknown constrained-input nonlinear systems

In this paper, an adaptive output feedback event-triggered optimal control algorithm is proposed for partially unknown constrained-input continuous-time nonlinear systems. First, a neural network observer is constructed to estimate unmeasurable state. Next, an event-triggered condition is established, and only when the event-triggered condition is violated will the event be triggered and the state be sampled. Then, an event-triggered-based synchronous integral reinforcement learning (ET-SIRL) control algorithm with critic-actor neural networks (NNs) architecture is proposed to solve the event-triggered Hamilton–Jacobi–Bellman equation under the established event-triggered condition. The critic and actor NNs are used to approximate cost function and optimal event-triggered optimal control law, respectively. Meanwhile, the event-triggered-based closed-loop system state and all the neural network weight estimation errors are uniformly ultimately bounded proved by Lyapunov stability theory, and there is no Zeno behavior. Finally, two numerical examples are presented to show the effectiveness of the proposed ET-SIRL control algorithm.     

Dynamic event-triggered control for a class of p-normal nonlinear time-delay systems via output feedback

In this article, the event-triggered (ET) output feedback control problem is discussed for a class of $p$-normal nonlinear time-delay systems. Different from the related existing literature, the linear growth condition is relaxed to the homogeneous one and the growth rate is unknown. In the case of unknown time-varying delays and control coefficients, nonsmooth control law and some extra redundant terms will be encountered in the design of ET controller, which would bring substantial challenges to the achievement of global convergence. To cope with unknown growth rate and time-varying delays, two dynamic gains and an appropriate Lyapunov–Krasovskii functional (LKF) are introduced. Then, a new dynamic ET mechanism is given, in which the triggering threshold can be tuned dynamically. It is proved that all the signals of the closed-loop system (CLS) are bounded. Corresponding examples are given to indicate the validity of the developed theoretical results.     

Leader-following consensus of nonlinear fractional-order multi-agent systems via event-triggered control

This paper investigates the consensus problem of leader-following multi-agent systems with fractional-order nonlinear dynamics. A typical event is defined as some error signals exceeding a given threshold. By applying Lyapunov functional approach and skills of computing function limit, consensus of the controlled multi-agent systems can be reached asymptotically. Meanwhile, the event-triggered algorithm can exclude Zeno behaviours. Finally, a numerical simulation is exploited to verify the effectiveness of the theoretical result.     

Dynamic event-triggered prescribed performance control for nonlinear systems with signal temporal logic

This article is concerned with the problem of dynamic event-triggered prescribed performance control for nonlinear systems under signal temporal logic tasks. By utilizing the method of prescribed performance control, the constrained plant can be transformed into an unconstrained one, and a dynamic event-triggered feedback control law is generated for the transformed system to ensure that the signal temporal logic specification is satisfied. A dynamic event-triggered mechanism is designed to guarantee the event-triggered stability, safety and complex specification. Besides, Zeno phenomenon is definitely avoided. Compared with the continuous-time feedback controller, the event-triggered controller has proven to be effective in reducing sensing, communication and computation costs. Finally, two simulations are given to illustrate the effectiveness of theoretical results.     

Event-triggered adaptive tracking control for nonlinear systems with input saturation and unknown control directions

This article is concerned with event-triggered adaptive tracking control design of strict-feedback nonlinear systems, which are subject to input saturation and unknown control directions. In the design procedure, a smooth nonlinear function is employed to approximate the saturation function so that the controller can be designed under the framework of backstepping. The Nussbaum gain technique is employed to address the issue of the unknown control directions. A predetermined time convergent performance function and a nonlinear mapping technique are introduced to guarantee that the tracking error can converge in the predetermined time with a fast convergence rate and a high accuracy. Then the event-triggered adaptive prescribed performance tracking control strategy is proposed, which not only ensures the boundedness of all the closed-loop signals and the convergence of tracking error but also reduces the communication burden from the controller to the actuator. At last, the simulation study further tests the availability of the proposed control strategy.     

Global adaptive stabilisation for nonlinear systems with unknown control directions and input disturbance

This paper addresses the global adaptive stabilisation via switching and learning strategies for a class of uncertain nonlinear systems. Remarkably, the systems in question simultaneously have unknown control directions, unknown input disturbance and unknown growth rate, which makes the problem in question challenging to solve and essentially different from those in the existing literature. To solve the problem, an adaptive scheme via switching and learning is proposed by skilfully integrating the techniques of backstepping design, adaptive learning and adaptive switching. One key point in the design scheme is the introduction of the learning mechanism, in order to compensate the unknown input disturbance, and the other one is the design of the switching mechanism, through tuning the design parameters online to deal with the unknown control directions, unknown bound and period of input disturbance and unknown growth rate. The designed controller guarantees that all the signals of the resulting closed-loop systems are bounded, and furthermore, the closed-loop system states globally converge to zero.     

含有未知控制方向的非线性时滞系统学习控制

本文对于一类含有未知控制方向及时滞的非线性参数化系统,设计了自适应迭代学习控制算法.在设计控制算法过程中采用了参数分离技术和信号置换思想来处理系统中出现的时滞项,Nussbaum增益技术解决未知控制方向等问题.为了对系统中出现的未知时变参数和时不变参数进行估计,分别设计了差分及微分参数学习律.然后通过构造的Lyapunov-Krasovskii复合能量函数给出了系统跟踪误差渐近收敛及闭环系统中所有信号有界的条件.最后通过一个仿真例子说明了控制器设计的有效性.     

Improved prescribed performance control for nonlinear systems with unknown control direction and input dead-zone

In view of the input dead-zone, unknown control direction and difficulty in satisfying the prescribed performance that suffered in practical systems, an improved prescribed performance-based adaptive control scheme is stressed for uncertain nonlinear systems in this paper. Firstly, by adopting a characteristic function, the input dead-zone is linearized to a model with bounded perturbation. To settle the “computation complexity” issue, an adaptive controller is built via command filter design method, where the fuzzy logic systems are introduced to approximate the unknown nonlinearities. Meanwhile, the Nussbaum function is brought in controller design to counter the hardship of unknown control direction. Besides, the tracking error can be restricted in the prescribed boundary in finite time with the improved performance function. The presented control approach can not only ensure the finite-time convergence property of tracking error and the boundedness of all signals in the closed-loop system, but also easily implement in engineering. Finally, the simulation examples confirm the validity of the designed control scheme.     

Event-triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs

In this article, the event-triggered optimal tracking control problem for multiplayer unknown nonlinear systems is investigated by using adaptive critic designs. By constructing a neural network (NN)-based observer with input–output data, the system dynamics of multiplayer unknown nonlinear systems is obtained. Subsequently, the optimal tracking control problem is converted to an optimal regulation problem by establishing a tracking error system. Then, the optimal tracking control policy for each player is derived by solving coupled event-triggered Hamilton-Jacobi (HJ) equation via a critic NN. Meanwhile, a novel weight updating rule is designed by adopting concurrent learning method to relax the persistence of excitation (PE) condition. Moreover, an event-triggering condition is designed by using Lyapunov's direct method to guarantee the uniform ultimate boundedness (UUB) of the closed-loop multiplayer systems. Finally, the effectiveness of the developed method is verified by two different multiplayer nonlinear systems.     

Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics

An online adaptive optimal control is proposed for continuous-time nonlinear systems with completely unknown dynamics, which is achieved by developing a novel identifier-critic-based approximate dynamic programming algorithm with a dual neural network (NN) approximation structure. First, an adaptive NN identifier is designed to obviate the requirement of complete knowledge of system dynamics, and a critic NN is employed to approximate the optimal value function. Then, the optimal control law is computed based on the information from the identifier NN and the critic NN, so that the actor NN is not needed. In particular, a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier NN and critic NN simultaneously, which converge to small neighbourhoods around their ideal values. The closed-loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the Lyapunov theory. The proposed adaptation algorithm is also improved to achieve finite-time convergence of the NN weights. Finally, simulation results are provided to exemplify the efficacy of the proposed methods.     

Adaptive consensus control of nonlinear systems with unknown control directions

In this paper, we investigate the adaptive consensus control for a class of nonlinear systems with different unknown control directions where communications among the agents are represented by a directed graph. Based on the backstepping technique, a fully distributed adaptive control approach is proposed without using global information of the topology. Meanwhile, a novel Nussbaum-type function is proposed to address the consensus control with unknown control directions. It is proved that boundedness of all closed-loop signals and asymptotic consensus tracking for all the agents' outputs are ensured. In simulation studies, a numerical example is illustrated to show the effectiveness of the control scheme.     

切换非线性系统的输出反馈周期事件触发控制

本文针对一类在任意切换信号作用下的切换非线性系统, 研究了其输出反馈周期事件触发控制问题. 所考 虑的非线性系统采用非严格反馈形式且含有未知时变控制系数. 在本文中, 仅利用采样时刻的系统输出. 为了估计 系统的不可量测的状态, 基于采样的系统输出构造了降维状态观测器. 为了减少通信资源的利用, 提出了一种新的 输出反馈周期事件触发策略, 该策略包含仅利用事件触发时刻的信息构造的输出反馈事件触发控制器以及仅在采 样时刻间歇性监测的离散事件触发机制. 通过选取可容许的采样周期及合适的公共Lyapunov函数, 证明了闭环系统 在任意切换下全局渐近稳定. 最后, 通过将本文中所给出的控制方案应用到数值算例中验证了其有效性.     

Event-triggered adaptive consensus control of nonlinear multi-agent systems with unknown control directions and actuator saturation

This article addresses the event-triggered adaptive consensus control of nonlinear multi-agent systems with unknown control direction and actuator saturation. A new robust adaptive control algorithm based on an event-triggered mechanism is designed. The smooth Lipschitz function approximates the saturated nonlinear function, while the Nussbaum function handles unknown control directions and residual terms. The event-triggered mechanism is designed to determine the time of communication, significantly reducing the communication burden. An additional estimator is utilized to deal with unknown parameters involved in neighbor dynamics and prevent information exchange to consistency errors between connected subsystems. The results show that all the signals of the closed-loop system are uniformly bounded, and the consensus tracking error converges to a bounded set. Meanwhile, Zeno's behavior is eliminated. Simulation results confirm the superiority of the proposed method.     

基于事件触发非线性多智能体系统的固定时间一致性

本文研究了在有向拓扑下,带有非线性动力学多智能体系统的固定时间一致性问题.提出了一种新的基于事件触发机制的非线性控制策略,对于每个智能体给出了基于状态信息的事件触发条件,当状态误差满足所给条件时才触发事件,能有效的减小系统的能量耗散和控制器的更新频次.利用Lyapunov稳定性理论和代数图论,证明在该控制策略下,多智能体系统在固定时间能实现领导跟随一致性,且不存在Zeno行为.相较于有限时间一致性策略,采用固定时间一致性策略系统的收敛时间不再依赖于系统的初始状态.最后,仿真实例验证了理论结果的有效性.     

基于事件驱动的多智能体强化学习研究

本文针对多智能体强化学习中存在的通信和计算资源消耗大等问题,提出了一种基于事件驱动的多智能体强化学习算法,侧重于事件驱动在多智能体学习策略层方面的研究。在智能体与环境的交互过程中,算法基于事件驱动的思想,根据智能体观测信息的变化率设计触发函数,使学习过程中的通信和学习时机无需实时或按周期地进行,故在相同时间内可以降低数据传输和计算次数。另外,分析了该算法的计算资源消耗,以及对算法收敛性进行了论证。最后,仿真实验说明了该算法可以在学习过程中减少一定的通信次数和策略遍历次数,进而缓解了通信和计算资源消耗。     

Observer-based event-triggered distributed model predictive control for a class of nonlinear interconnected systems

In this paper, an observer-based event-triggered distributed model predictive control method is proposed for a class of nonlinear interconnected systems with bounded disturbances, considering unmeasurable states. First of all, the state observer is constructed. It is proved that the observation error is bounded. Second, distributed model predictive controller is designed by using observed value. Meanwhile, the event-triggered mechanism is set by using the error between the actual output and the predicted output. The setting of event-triggered mechanism not only ensures the error between the actual output and the predicted output within a certain range, but also reduces the calculation amounts of solving the optimization problem. The states of each subsystem enter the terminal invariant set by distributed model predictive control, and then are stabilized in the invariant set under the action of output feedback control law. In addition, sufficient conditions are given to ensure the feasibility of the algorithm and the stability of the closed-loop system. Finally, the numerical example is given, and the simulation results verify the effectiveness of the proposed algorithm.     

Adaptive event-triggered tracking control for a class of stochastic nonlinear systems with full-state constraints

This paper proposes an adaptive event trigger-based sample-and-hold tracking control scheme for a class of strict-feedback nonlinear stochastic systems with full-state constraints. By introducing a tan-type stochastic Barrier Lyapunov function (SBLF) combined with radial basis function neural networks (RBFNNs), which is used to approximate the nonlinear functions in backstepping procedures, an adaptive event-triggered controller is designed. It is shown with stochastic stability theory that all the states cannot violate their constraints, and Zeno behavior is excluded almost surely. Meanwhile, all the signals of the closed-loop systems are bounded almost surely and the tracking error converges to an arbitrary small compact set in the fourth-moment sense. A simulation example is given to show the effectiveness of the control scheme.     

Adaptive iterative learning control for nonlinearly parameterized systems with unknown time-varying delay and unknown control direction

This paper proposes a new adaptive iterative learning control approach for a class of nonlinearly parameterized systems with unknown time-varying delay and unknown control direction.By employing the parameter separation technique and signal replacement mechanism,the approach can overcome unknown time-varying parameters and unknown time-varying delay of the nonlinear systems.By incorporating a Nussbaum-type function,the proposed approach can deal with the unknown control direction of the nonlinear systems.Based on a Lyapunov-Krasovskii-like composite energy function,the convergence of tracking error sequence is achieved in the iteration domain.Finally,two simulation examples are provided to illustrate the feasibility of the proposed control method.