非线性主动悬架系统自适应最优控制

针对非线性主动悬架系统多性能指标综合优化问题,提出一类自适应最优控制方法.首先,通过引入一阶低通滤波操作,利用系统输入输出构建结构简单且调节参数少的一类未知非线性动态估计器,在线估计系统未知非线性动态;其次,构建包含乘驾舒适度、悬架行程空间及输入能耗的性能指标函数,采用单层神经网络对最优性能指标函数进行在线逼近,并得到新的哈密尔顿函数;为实现在线求解,构建一类新的基于参数估计误差信息的自适应律,在线更新神经网络权值并计算最优控制律;最后,理论分析闭环系统稳定性和收敛性,并通过专业软件Carsim与Matlab/Simulink搭建的联合仿真平台给出的对比仿真结果,验证所提出方法可有效解决主动悬架系统多目标性能优化控制问题,提升主动悬架系统综合性能.     

基于自适应动态规划的一类带有时滞的离散时间非线性系统的最优控制策略

针对一类状态和控制变量均带有时滞的非线性系统的带有二次性能指标函数最优控制问题, 本文提出了一种基于新的迭代自适应动态规划算法的最优控制方案. 通过引进时滞矩阵函数, 应用动态规划理论, 本文获得了最优控制的显式表达式, 然后通过自适应评判技术获得最优控制量. 本文给出了收敛性证明以保证性能指标函数收敛到最优. 为了实现所提出的算法, 本文采用神经网络近似性能指标函数、计算最优控制策略、求解时滞矩阵函数、以及给非线性系统建模. 最后本文给出了两个仿真例子说明所提出的最优策略的有效性.     

基于联合滤波算法的复杂时变系统在线自适应逆控制

针对一类结构和参数均具备时变特性的复杂时变系统,提出一种新的基于联合滤波算法的在线自适应逆控制方法.该方法在处理参数时变问题的同时可兼顾系统的结构时变特性,实现复杂动态系统的在线跟踪控制.同时提出新的联合Volterra核函数滤波算法,该算法克服了原Volterra滤波器计算复杂运算速度慢的缺点,实现了动态非线性系统的在线跟踪控制.通过仿真分析可以得出,对于此类线性、非线性复杂时变系统,基于新的联合滤波器的自适应逆控制方法可以快速有效的实现动态对象在线建模与控制.     

基于伪谱法的自由采样实时最优反馈控制及应用

利用高斯伪谱法收敛速率快、精度高的特点,基于通用伪谱优化软件包在线求解非线性系统的最优控制问题.将伪谱反馈控制理论与非线性最优控制理论结合起来,给出了一种自由采样实时最优反馈控制算法,该算法通过连续在线生成开环最优控制的方式提供闭环反馈.考虑计算误差、模型参数不确定性和干扰的作用,假定系统状态方程右侧的非线性向量函数关于状态、控制和系统参数是Lipschitz连续的,利用Bellman最优性原理对闭环控制系统的有界稳定性进行了分析和理论证明.最后,以高超声速再入飞行器为应用对象,研究了其再入制导问题,仿真结果验证了该算法的可行性和有效性.     

基于数据ADP算法的一类带有执行器饱和的未知离散时间系统最优跟踪控制

针对一类带有执行器饱和的未知动态离散时间非线性系统, 提出了一种新的最优跟踪控制方案. 该方案基于迭代自适应动态规划算法, 为了实现最优控制, 首先建立了未知系统动态的数据辨识器. 通过引入M网络, 获得了稳态控制的精确表达式. 为了消除执行器饱和的影响, 提出了一个非二次的性能指标函数. 然后提出了一种迭代自适应动态规划算法获得最优跟踪控制的解, 并给出了收敛性分析. 为了实现最优控制方案, 神经网络被用来构建数据辨识器、计算性能指标函数、近似最优控制策略和求解稳态控制. 仿真结果验证了本文所提出的最优跟踪控制方法的有效性.     

基于ADP算法的带时滞及饱和的非线性系统最优控制

针对控制时滞及带饱和的一类离散时间非线性系统的最优控制问题,通过重构性能指标函数和对应的系统变换,处理了性能指标函数中的控制耦合项;继而引入一个合适的泛函,解决了控制带饱和问题.给出了一个新的性能指标函数,利用迭代自适应动态规划(ADP)算法获得最优控制.为实现该算法,采用神经网络逼近函数来求解最优控制问题.仿真结果验证了方法的有效性.     

基于整体辨识策略的非线性自适应控制方法

针对一类离散时间下的未知动态非线性系统,为解决传统自适应控制方法在交替辨识非线性系统时由于辨识精度低而导致的控制性能差的问题,本文提出了一种基于整体辨识策略的未建模动态补偿的自适应控制方法.利用随机向量函数链接(RVFL)网络的直链与增强结构特性挖掘其与低阶线性模型和高阶未建模动态项的等价对应关系,并融入权值偏差惩罚项,设计了网络模型参数在线更新算法辨识非线性系统参数.根据在线辨识的线性模型参数和未建模动态估计量,采用一步超前最优控制策略设计线性控制器和未建模动态补偿器.数值仿真表明,所提方法优于交替辨识下的非线性自适应控制方法,并通过工业应用的仿真研究验证所提方法在工业上的可用性.最后,对本文控制方法在实际应用中的潜在问题及理论受限条件的放松进行分析和展望.     

模糊/神经自适应控制及其在非线性系统中的应用

针对连续未知非线性系统,提出一种基于观测器并保证稳定性和有界性的自适应模糊历中经算法。本算法利用T—S模糊系统或者神经网络径向基函数构成间接自适应控制器,其参数根据控制率和自适应率进行在线调整,并利用Lyapunov综合法确保对非线性环节渐进跟踪的稳定性。最后,通过对倒立摆系统的仿真,证明该算法在非线性系统控制中应用的可行性。     

带ε误差限的近似最优控制

近似动态规划方法求解非线性系统最优控制, 需要迭代无限步才能得到最优控制律. 本文提出了一种ε–近似最优控制算法, 选择ε误差限, 通过自适应迭代不断逼近哈密顿– 雅可比– 贝尔曼(HJB)方程的解, 应用神经网络实现在有限步迭代后得到带ε误差限的近似最优控制律. 计算机仿真结果表明了该算法的有效性.     

Approximate optimal control with εerror bound

近似动态规划方法求解非线性系统最优控制,需要迭代无限步才能得到最优控制律．本文提出了一种ε-近似最优控制算法,选择ε误差限,通过自适应迭代不断逼近哈密顿-雅可比-贝尔曼（HJB）方程的解,应用神经网络实现在有限步迭代后得到带ε误差限的近似最优控制律．计算机仿真结果表明了该算法的有效性．     

Online Adaptive Approximate Optimal Tracking Control with Simplified Dual Approximation Structure for Continuous-time Unknown Nonlinear Systems

This paper proposes an online adaptive approximate solution for the infinite-horizon optimal tracking control problem of continuous-time nonlinear systems with unknown dynamics. The requirement of the complete knowledge of system dynamics is avoided by employing an adaptive identifier in conjunction with a novel adaptive law, such that the estimated identifier weights converge to a small neighborhood of their ideal values. An adaptive steady-state controller is developed to maintain the desired tracking performance at the steady-state, and an adaptive optimal controller is designed to stabilize the tracking error dynamics in an optimal manner. For this purpose, a critic neural network (NN) is utilized to approximate the optimal value function of the Hamilton-Jacobi-Bellman (HJB) equation, which is used in the construction of the optimal controller. The learning of two NNs, i.e., the identifier NN and the critic NN, is continuous and simultaneous by means of a novel adaptive law design methodology based on the parameter estimation error. Stability of the whole system consisting of the identifier NN, the critic NN and the optimal tracking control is guaranteed using Lyapunov theory; convergence to a near-optimal control law is proved. Simulation results exemplify the effectiveness of the proposed method.      

Observer-based Adaptive Optimal Control for Unknown Singularly Perturbed Nonlinear Systems With Input Constraints

This paper introduces an observer-based adaptive optimal control method for unknown singularly perturbed nonlinear systems with input constraints. First, a multi-time scales dynamic neural network (MTSDNN) observer with a novel updating law derived from a properly designed Lyapunov function is proposed to estimate the system states. Then, an adaptive learning rule driven by the critic NN weight error is presented for the critic NN, which is used to approximate the optimal cost function. Finally, the optimal control action is calculated by online solving the Hamilton-Jacobi-Bellman (HJB) equation associated with the MTSDNN observer and critic NN. The stability of the overall closed-loop system consisting of the MTSDNN observer, the critic NN and the optimal control action is proved. The proposed observer-based optimal control approach has an essential advantage that the system dynamics are not needed for implementation, and only the measured input/output data is needed. Moreover, the proposed optimal control design takes the input constraints into consideration and thus can overcome the restriction of actuator saturation. Simulation results are presented to confirm the validity of the investigated approach.      

基于梯度估计的非线性系统最优控制及仿真

基于自适应动态规划（ADP）执行-评价结构,应用神经网络（NN）对非线性系统进行最优控制求解.首先提出所求解非线性系统的一般形式;其次给定二次正定性能指标,求其哈密尔顿函（HJB）函数;分别应用神经网络对执行-评价结构中的性能指标和最优控制进行逼近,神经网络权重参数应用梯度法求得,从而可以求得其最有控制策略.而且对执行机构和评价机构神经网络权重参数的收敛性以及系统总体的稳定性进行了详细的分析,证明所求控制策略可以使系统稳定;最后,用仿真结果来验证所提出的方法的可行性.     

自适应动态规划综述

自适应动态规划(Adaptive dynamic programming, ADP)是最优控制领域新兴起的一种近似最优方法, 是当前国际最优化领域的研究热点. ADP方法 利用函数近似结构来近似哈密顿--雅可比--贝尔曼(Hamilton-Jacobi-Bellman, HJB)方程的解, 采用离线迭代或者在线更新的方法, 来获得系统的近似最优控制策略, 从而能够有效地解决非线性系统的优化控制问题. 本文按照ADP的结构变化、算法的发展和应用三个方面介绍ADP方法. 对目前ADP方法的研究成果加以总结, 并对这 一研究领域仍需解决的问题和未来的发展方向作了进一步的展望.     

Online adaptive optimal control for continuous-time nonlinear systems with completely unknown dynamics

An online adaptive optimal control is proposed for continuous-time nonlinear systems with completely unknown dynamics, which is achieved by developing a novel identifier-critic-based approximate dynamic programming algorithm with a dual neural network (NN) approximation structure. First, an adaptive NN identifier is designed to obviate the requirement of complete knowledge of system dynamics, and a critic NN is employed to approximate the optimal value function. Then, the optimal control law is computed based on the information from the identifier NN and the critic NN, so that the actor NN is not needed. In particular, a novel adaptive law design method with the parameter estimation error is proposed to online update the weights of both identifier NN and critic NN simultaneously, which converge to small neighbourhoods around their ideal values. The closed-loop system stability and the convergence to small vicinity around the optimal solution are all proved by means of the Lyapunov theory. The proposed adaptation algorithm is also improved to achieve finite-time convergence of the NN weights. Finally, simulation results are provided to exemplify the efficacy of the proposed methods.     

Fuzzy adaptive dynamic programming-based optimal leader-following consensus for heterogeneous nonlinear multi-agent systems

In this paper, a novel online iterative scheme, based on fuzzy adaptive dynamic programming, is proposed for distributed optimal leader-following consensus of heterogeneous nonlinear multi-agent systems under directed communication graph. This scheme combines game theory, adaptive dynamic programming together with generalized fuzzy hyperbolic model (GFHM). Firstly, based on precompensation technique, an appropriate model transformation is proposed to convert the error system into augmented error system, and an exquisite performance index function is defined for this system. Secondly, on the basis of Hamilton–Jacobi–Bellman (HJB) equation, the optimal consensus control is designed and a novel policy iteration (PI) algorithm is put forward to learn the solutions of the HJB equation online. Here, the proposed PI algorithm is implemented on account of GFHMs. Compared with dual-network model including critic network and action network, the proposed scheme only requires critic network. Thirdly, the augmented consensus error of each agent and the weight estimation error of each GFHM are proved to be uniformly ultimately bounded, and the stability of our method has been verified. Finally, some numerical examples and application examples are conducted to demonstrate the effectiveness of the theoretical results.

     

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming (ADP) where only one critic neural network (NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness (UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.      

Optimal PID Controller Autotuning Design for MIMO Nonlinear Systems Based on the Adaptive SLP Algorithm

In this paper, an adaptive swarm learning process (SLP) algorithm for designing the optimal proportional integral and derivative (PID) parameter for a multiple-input multiple-output (MIMO) control system is proposed. The SLP algorithm is proposed to improve the performance and convergence of PID parameter autotuning by applying the swarm algorithm and the learning process. The adaptive SLP algorithm improves the stability, performance and robustness of the traditional SLP algorithm to apply it to a MIMO control system. It can update the online weights of the SLP algorithm caused by the errors in the settling time, rise time and overshoot of the system based on a stable learning rate. The gradient descent is applied to update the weights. The stable learning rate is verified based on the Lyapunov stability theorem. Additionally, simulations are performed to verify the superiority of the algorithm in terms of performance and robustness. Results that compare the adaptive SLP algorithm with the traditional SLP, a neural network (NN), the genetic algorithm (GA), the particle swarm and optimization (PSO) algorithm and the kidney-inspired algorithm (KIA) based on a two-wheel inverted pendulum system are presented. With respect to performance and robustness, the adaptive SLP algorithm provides a better response than the traditional SLP, NN, GA, PSO and KIA.

     

Neural-based online finite-time optimal tracking control for wheeled mobile robotic system with inequality constraints

In this study, a finite-time online optimal controller was designed for a nonlinear wheeled mobile robotic system (WMRS) with inequality constraints, based on reinforcement learning (RL) neural networks. In addition, an extended cost function, obtained by introducing a penalty function to the original long-time cost function, was proposed to deal with the optimal control problem of the system with inequality constraints. A novel Hamilton-Jacobi-Bellman (HJB) equation containing the constraint conditions was defined to determine the optimal control input. Furthermore, two neural networks (NNs), a critic and an actor NN, were established to approximate the extended cost function and the optimal control input, respectively. The adaptation laws of the critic and actor NN were obtained with the gradient descent method. The semi-global practical finite-time stability (SGPFS) was proved using Lyapunov's stability theory. The tracking error converges to a small region near zero within the constraints in a finite period. Finally, the effectiveness of the proposed optimal controller was verified by a simulation based on a practical wheeled mobile robot model.