非线性离散时间系统带ε误差限的自适应动态规划

为了获得非线性离散时间系统的最优控制策略,基于自适应动态规划的原理,提出了一种带误差限的自适应动态规划方法.对于一个任意的状态,用一个有限长度的控制序列近似最优控制序列,使性能指标与最优性能指标的误差在一个较小的范围内.选取一个非线性离散时间系统对算法的性能进行数值实验,结果验证了该算法的有效性,用较少的计算代价获得了近似最优的控制策略.     

Constrained Optimal Control of Hybrid Systems With a Linear Performance Index

We consider the constrained finite and infinite time optimal control problem for the class of discrete-time linear hybrid systems. When a linear performance index is used the finite and infinite time optimal solution is a piecewise affine state feedback control law. In this paper, we present algorithms that compute the optimal solution to both problems in a computationally efficient manner and with guaranteed convergence and error bounds. Both algorithms combine a dynamic programming exploration strategy with multiparametric linear programming and basic polyhedral manipulation     

具有最优动态性能的鲁棒镇定控制器设计

针对ＳＩＳＯ线性离散系统,利用线性规划方法设计具有指定最优动态性能的鲁棒稳定控制器。当线性离散模型的零,极点已知时,将最优动态性能指标在指定输入信号下直接转化为线性规划问题。从而解出最优响应输出序列。最优动态性能指标与鲁棒稳定性的统一使该控制器的设计方法具备了工业应用条件。仿真实例验证了结果的正确性。     

Dynamic surface control for discrete-time strict-feedback systems with network-induced delays using predictive event-triggered strategy

In this paper, a novel dynamic surface control methodology with a predictive event-triggered strategy is proposed for discrete-time strict-feedback systems. In order to evaluate immeasurable state variables, a least square evaluation method is adopted. A dynamic surface controller is designed for discrete-time strict-feedback systems. When the transmission time of Sensor-to-Controller (S–C) or Controller-to-Actuator (C–A) channel increases, the stability and dynamic performance of the system deteriorates are improved by the proposed predictive event-triggered control strategy. Furthermore, the proposed methodology decreases the use of network resources while the whole system still keeps stable and exhibits an acceptable dynamic performance. The simulation results demonstrate that the proposed methodology is effective.     

Neural-network-based stochastic linear quadratic optimal tracking control scheme for unknown discrete-time systems using adaptive dynamic programming

In this paper, a stochastic linear quadratic optimal tracking scheme is proposed for unknown linear discrete-time (DT) systems based on adaptive dynamic programming (ADP) algorithm. First, an augmented system composed of the original system and the command generator is constructed and then an augmented stochastic algebraic equation is derived based on the augmented system. Next, to obtain the optimal control strategy, the stochastic case is converted into the deterministic one by system transformation, and then an ADP algorithm is proposed with convergence analysis. For the purpose of realizing the ADP algorithm, three back propagation neural networks including model network, critic network and action network are devised to guarantee unknown system model, optimal value function and optimal control strategy, respectively. Finally, the obtained optimal control strategy is applied to the original stochastic system, and two simulations are provided to demonstrate the effectiveness of the proposed algorithm.     

Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach

In this paper, a finite-horizon neuro-optimal tracking control strategy for a class of discrete-time nonlinear systems is proposed. Through system transformation, the optimal tracking problem is converted into designing a finite-horizon optimal regulator for the tracking error dynamics. Then, with convergence analysis in terms of cost function and control law, the iterative adaptive dynamic programming (ADP) algorithm via heuristic dynamic programming (HDP) technique is introduced to obtain the finite-horizon optimal tracking controller which makes the cost function close to its optimal value within an ?-error bound. Three neural networks are used as parametric structures to implement the algorithm, which aims at approximating the cost function, the control law, and the error dynamics, respectively. Two simulation examples are included to complement the theoretical discussions.     

U-D factorisation of the strengthened discrete-time optimal projection equations

Algorithms for optimal reduced-order dynamic output feedback control of linear discrete-time systems with white stochastic parameters are U-D factored in this paper. U-D factorisation enhances computational accuracy, stability and possibly efficiency. Since U-D factorisation of algorithms for optimal full-order output feedback controller design was recently published by us, this paper focusses on the U-D factorisation of the optimal oblique projection matrix that becomes part of the solution as a result of order-reduction. The equations producing the solution are known as the optimal projection equations which for discrete-time systems have been strengthened in the past. The U-D factored strengthened discrete-time optimal projection equations are presented in this paper by means of a transformation that has to be applied recursively until convergence. The U-D factored and conventional algorithms are compared through a series of examples.     

基于迭代神经动态规划的数据驱动非线性近似最优调节

利用数据驱动控制思想,建立一种设计离散时间非线性系统近似最优调节器的迭代神经动态规划方法.提出针对离散时间一般非线性系统的迭代自适应动态规划算法并且证明其收敛性与最优性.通过构建三种神经网络,给出全局二次启发式动态规划技术及其详细的实现过程,其中执行网络是在神经动态规划的框架下进行训练.这种新颖的结构可以近似代价函数及其导函数,同时在不依赖系统动态的情况下自适应地学习近似最优控制律.值得注意的是,这在降低对于控制矩阵或者其神经网络表示的要求方面,明显地改进了迭代自适应动态规划算法的现有结果,能够促进复杂非线性系统基于数据的优化与控制设计的发展.通过两个仿真实验,验证本文提出的数据驱动最优调节方法的有效性.     

含状态和输入时滞的离散时间系统的近似最优跟踪控制

针对状 态和控制输入均含有时滞的离散时间系统, 提出最优跟踪控制的设计方法. 通 过引入一种新的状态向量, 将含有状态和控制输入时滞的离散时间系统转化为 含有虚拟扰动项的无时滞离散时间系统. 根据最优控制理论, 构造离散Riccati矩阵方 程和离散Stein矩阵方程的序列, 并证明该解序列一致收敛于变换后的离散时间系统的最优跟 踪控制策略. 利用最优控制的逐次逼近设计方法, 得到最优跟踪控制的近似 解, 并给出求解最优跟踪控制律的算法. 仿真算例表明了所提出最优跟踪控制 方法的有效性.     

Optimal Semicomputable Approximations to Reachable and Invariant Sets

In this paper we consider the computation of reachable, viable and invariant sets for discrete-time systems. We use the framework of type-two effectivity, in which computations are performed by Turing machines with infinite input and output tapes, with the representations of computable topology. We see that the reachable set is lower-semicomputable, and the viability and invariance kernels are upper-semicomputable. We then define an upper-semicomputable over-approximation to the reachable set, and lower-semicomputable under-approximations to the viability and invariance kernels, and show that these approximations are optimal.     

基于非策略Q-学习的网络控制系统最优跟踪控制

针对具有数据包丢失的网络化控制系统跟踪控制问题,提出一种非策略Q-学习方法,完全利用可测数据,在系统模型参数未知并且网络通信存在数据丢失的情况下,实现系统以近似最优的方式跟踪目标.首先,刻画具有数据包丢失的网络控制系统,提出线性离散网络控制系统跟踪控制问题;然后,设计一个Smith预测器补偿数据包丢失对网络控制系统性能的影响,构建具有数据包丢失补偿的网络控制系统最优跟踪控制问题;最后,融合动态规划和强化学习方法,提出一种非策略Q-学习算法.算法的优点是:不要求系统模型参数已知,利用网络控制系统可测数据,学习基于预测器状态反馈的最优跟踪控制策略;并且该算法能够保证基于Q-函数的迭代Bellman方程解的无偏性.通过仿真验证所提方法的有效性.     

基于强化学习的部分线性离散时间系统的最优输出调节

针对同时具有线性外部干扰与非线性不确定性下的离散时间部分线性系统的最优输出调节问题, 提出了仅利用在线数据的基于强化学习的数据驱动控制方法. 首先, 该问题可拆分为一个受约束的静态优化问题和一个动态规划问题, 第一个问题可以解出调节器方程的解. 第二个问题可以确定出控制器的最优反馈增益. 然后, 运用小增益定理证明了存在非线性不确定性离散时间部分线性系统的最优输出调节问题的稳定性. 针对传统的控制方法需要准确的系统模型参数用来解决这两个优化问题, 提出了一种数据驱动离线策略更新算法, 该算法仅使用在线数据找到动态规划问题的解. 然后, 基于动态规划问题的解, 利用在线数据为静态优化问题提供了最优解. 最后, 仿真结果验证了该方法的有效性.     

离散时间最优控制—–评论动态规划

阐述离散时间最优控制的特点.对比3种求解离散时间最优控制的解法,即:1)用非线性规划求解离散时间最优控制;2)用无约束优化求解离散时间最优控制;3)动态规划及其数值解.1)和2)都适用于多维静态优化,计算效率较高,是高级方法.在名义上,3)为动态优化.实际上,3)为一维分段无约束静态优化,计算效率较低,是初级方法.本文并用数字实例进一步阐明动态规划及其数值解在求解方面较差,故动态规划及其数值解已失去实用价值.在求解离散时间最优控制问题方面,无法与非线性规划求解相匹敌.     

切换自治系统的动态输出反馈镇定

本文研究离散时间切换线性自治系统的输出反馈镇定问题. 在切换系统可观测的假设下, 设计具有多线性 时变增益的动态观测器, 实现有限时间状态估计. 在此基础上, 设计多路径动态输出反馈切换策略, 实现闭环系统指 数收敛.     

Finite horizon quadratic optimal control and a separation principle for Markovian jump linear systems

In this note, we consider the finite-horizon quadratic optimal control problem of discrete-time Markovian jump linear systems driven by a wide sense white noise sequence. We assume that the output variable and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system minimizes the quadratic functional cost of the system over a finite horizon period of time. As in the case with no jumps, we show that an optimal controller can be obtained from two coupled Riccati difference equations, one associated to the optimal control problem when the state variable is available, and the other one associated to the optimal filtering problem. This is a principle of separation for the finite horizon quadratic optimal control problem for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the linear quadratic Gaussian control of discrete-time linear systems.     

<Emphasis Type="BoldItalic">H</Emphasis><Subscript>2</Subscript>-Control and the Separation Principle for Discrete-Time Markovian Jump Linear Systems

In this paper we consider the H₂-control problem of discrete-time Markovian jump linear systems. We assume that only an output and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system is mean square stable and minimizes the H₂-norm of the system. As in the case with no jumps, we show that an optimal controller can be obtained from two sets of coupled algebraic Riccati equations, one associated with the optimal control problem when the state variable is available, and the other associated with the optimal filtering problem. This is the principle of separation for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the H₂-control of discrete-time linear systems. Date received: June 1, 2001. Date revised: October 13, 2003.     

Analysis of a class of discrete-time systems with power rule

In this communique, the properties of a class of discrete-time dynamic systems with power rule are studied and comparisons with their continuous-time counterparts as well as the linear discrete-time systems are made. It is shown that using power rule can be beneficial for improving dynamic behaviors of discrete-time systems. For example, a power control law may stabilize a discrete-time system which is not stabilizable by the linear control law.     

A neural-network-based iterative GDHP approach for solving a class of nonlinear optimal control problems with control constraints

In this paper, a novel neural-network-based iterative adaptive dynamic programming (ADP) algorithm is proposed. It aims at solving the optimal control problem of a class of nonlinear discrete-time systems with control constraints. By introducing a generalized nonquadratic functional, the iterative ADP algorithm through globalized dual heuristic programming technique is developed to design optimal controller with convergence analysis. Three neural networks are constructed as parametric structures to facilitate the implementation of the iterative algorithm. They are used for approximating at each iteration the cost function, the optimal control law, and the controlled nonlinear discrete-time system, respectively. A simulation example is also provided to verify the effectiveness of the control scheme in solving the constrained optimal control problem.     

Partially observed non-linear risk-sensitive optimal stopping control for non-linear discrete-time systems

In this paper we introduce and solve the partially observed optimal stopping non-linear risk-sensitive stochastic control problem for discrete-time non-linear systems. The presented results are closely related to previous results for finite horizon partially observed risk-sensitive stochastic control problem. An information state approach is used and a new (three-way) separation principle established that leads to a forward dynamic programming equation and a backward dynamic programming inequality equation (both infinite dimensional). A verification theorem is given that establishes the optimal control and optimal stopping time. The risk-neutral optimal stopping stochastic control problem is also discussed.     

Dynamic programming for constrained optimal control of discrete-time linear hybrid systems

In this paper we study the solution to optimal control problems for constrained discrete-time linear hybrid systems based on quadratic or linear performance criteria. The aim of the paper is twofold. First, we give basic theoretical results on the structure of the optimal state-feedback solution and of the value function. Second, we describe how the state-feedback optimal control law can be constructed by combining multiparametric programming and dynamic programming.