Discrete-time optimal control for stochastic nonlinear polynomial systems

This paper presents a solution to the discrete-time optimal control problem for stochastic nonlinear polynomial systems over linear observations and a quadratic criterion. The solution is obtained in two steps: the optimal control algorithm is developed for nonlinear polynomial systems by considering complete information when generating a control law. Then, the state estimate equations for discrete-time stochastic nonlinear polynomial system over linear observations are employed. The closed-form solution is finally obtained substituting the state estimates into the obtained control law. The designed optimal control algorithm can be applied to both distributed and lumped systems. To show effectiveness of the proposed controller, an illustrative example is presented for a second degree polynomial system. The obtained results are compared to the optimal control for the linearized system.     

On a general optimal algorithm for multirate output feedbackcontrollers for linear stochastic periodic systems

A modified optimal algorithm for multirate output feedback controllers of linear stochastic periodic systems is developed. By combining the discrete-time linear quadratic regulation (LQR) control problem and the discrete-time stochastic linear quadratic regulation (SLQR) control problem to obtain an extended linear quadratic regulation (ELQR) control problem, one derives a general optimal algorithm to balance the advantages of the optimal transient response of the LQR control problem and the optimal steady-state regulation of the SLQR control problem. In general, the solution of this algorithm is obtained by solving a set of coupled matrix equations. Special cases for which the coupled matrix equations can be reduced to a discrete-time algebraic Riccati equation are discussed. A reducable case is the optimal algorithm derived by H.M. Al-Rahmani and G.F. Franklin (1990), where the system has complete state information and the discrete-time quadratic performance index is transformed from a continuous-time one     

基于双线性模型的连续时间非线性最优控制的DISOPE 算法

对连续时间非线性最优控制问题出了基于双线性二次型问题的ＤＩＳＯＰＥ算法,在模型与实际存在差异的情况下,通过求解修正的基于双线性模型的优化总理２和参数估计问题,给出了实际问题的最优解,提出了求解非齐次双线性二次型问题的迭代算法,分析了该算法的收敛性,仿真结果表明该算法比现有算法有更好的收敛特性。     

具有模型和实际差异的非线性离散动态系统最优控制

针对模型与实际存在一定差异的情况,提出了一种求解非 线性离散系统最段控制的动态系统优化和参数估计集成（Ｄｙｎａｍｉｃ Ｉｎｔｅｇｒａｔｅｄ Ｓｙｓｔｅｍ Ｏｐｔｉｍｉｚａｔｉｏｎ Ｐａｒａｍｅｔｅｒ Ｅｓｔｉｍａｔｉｏｎ,简称ＤＩＳＯＰＥ）的研究法。推导出一组求解一类有终端等式约束的非齐次的线性两点边值问题的递推公式,对于有终端等式约束的非线线性最优控制问题得到了在计算上易于实现的一种ＤＩＳＯＰＥ     

基于信息融合最优估计的非线性离散系统预测控制

针对非线性离散系统的二次型最优预测控制问题, 提出了一种基于信息融合最优估计的迭代预测控制算法. 通过融合二次型性能指标函数中包含的未来参考轨迹和控制能量的软约束信息, 以及系统状态方程和输出方程的硬约束信息, 获得协状态序列和控制序列的最优估计. 通过二自由度机器人操作手的转移控制仿真, 表明了该控制算法具有良好的稳定性和鲁棒性.     

离散双输出反馈控制的最优协调性合成设计

本文以离散二次型最优控制理论为基础，研究了线性时不变系统在双输出反馈控制情况下控制作用的协调性合成设计问题，构造了一个数值迭代算法来求解相应的最侉 控制规律。通过产例计算和对比分析，既证明了该方法的收敛性和有效性又说明了这种控制问题的一些本质特征。     

Data-driven optimal terminal iterative learning control

This paper presents a data-driven optimal terminal iterative learning control (TILC) approach for linear and nonlinear discrete-time systems. The iterative learning control law is updated from only terminal output tracking error instead of entire output trajectory tracking error. The only required knowledge of a controlled system is that the Markov matrices of linear systems or the partial derivatives of nonlinear systems with respect to control inputs are bounded. Rigorous analysis and convergence proof are developed with sufficient conditions for the terminal ILC design and the results are developed for both linear and nonlinear discrete-time systems. Simulation results illustrate the applicability and effectiveness of the proposed approach.     

非线性离散系统的近似最优跟踪控制

研究非线性离散系统的最优跟踪控制问题. 通过在由最优控制问题所导致的非线性两点边值问题中引入灵敏度参数, 并对它进行Maclaurin级数展开, 将原最优跟踪控制问题转化为一族非齐次线性两点边值问题. 得到的最优跟踪控制由解析的前馈反馈项和级数形式的补偿项组成. 解析的前馈反馈项可以由求解一个Riccati差分方程和一个矩阵差分方程得到. 级数补偿项可以由一个求解伴随向量的迭代算法近似求得. 以连续槽式反应器为例进行仿真验证了该方法的有效性．     

输入约束不确定系统的点对点迭代学习控制与优化

为解决工业过程中机械臂等特殊重复运行系统的输出在有限时间内无需实现全轨迹跟踪,仅需跟踪期望轨迹上某些特殊关键点的控制问题,针对线性时不变离散系统提出一种基于范数最优的点对点迭代学习控制算法.通过输入输出时间序列矩阵模型变换构建综合性多目标点性能指标函数,求解二次型最优解得到优化迭代学习控制律,同时给出模型标称和不确定情形下最大奇异值形式鲁棒控制算法收敛的充分条件,并进一步推广得到输入约束系统优化控制算法的收敛性结果,最后在三轴龙门机器人模型上验证算法的有效性.     

A new neural network-based approach for self-tuning control of nonlinear SISO discrete-time systems

This article presents a new neural network-based approach for self-tuning control of nonlinear single-input single-output (SISO) discrete-time dynamic systems. According to the approach, a neural network ARMAX (NN-ARMAX) model of the system is identified and continuously updated, using an online training algorithm. Control design is accomplished by solving an optimal discrete-time linear quadratic tracking problem using an observer-type linear state-space Kalman innovation model, which is built from the parameters of a local linear version of the NN-ARMAX model. The state-feedback control law is implemented using the Kalman state, which is calculated without estimating the noise covariance properties. The proposed control approach is shown to be very effective and outperforms the self-tuning control approach based on a linear ARMAX model on two simulation examples.     

A neural-network-based iterative GDHP approach for solving a class of nonlinear optimal control problems with control constraints

In this paper, a novel neural-network-based iterative adaptive dynamic programming (ADP) algorithm is proposed. It aims at solving the optimal control problem of a class of nonlinear discrete-time systems with control constraints. By introducing a generalized nonquadratic functional, the iterative ADP algorithm through globalized dual heuristic programming technique is developed to design optimal controller with convergence analysis. Three neural networks are constructed as parametric structures to facilitate the implementation of the iterative algorithm. They are used for approximating at each iteration the cost function, the optimal control law, and the controlled nonlinear discrete-time system, respectively. A simulation example is also provided to verify the effectiveness of the control scheme in solving the constrained optimal control problem.     

基于强化学习的部分线性离散时间系统的最优输出调节

针对同时具有线性外部干扰与非线性不确定性下的离散时间部分线性系统的最优输出调节问题, 提出了仅利用在线数据的基于强化学习的数据驱动控制方法. 首先, 该问题可拆分为一个受约束的静态优化问题和一个动态规划问题, 第一个问题可以解出调节器方程的解. 第二个问题可以确定出控制器的最优反馈增益. 然后, 运用小增益定理证明了存在非线性不确定性离散时间部分线性系统的最优输出调节问题的稳定性. 针对传统的控制方法需要准确的系统模型参数用来解决这两个优化问题, 提出了一种数据驱动离线策略更新算法, 该算法仅使用在线数据找到动态规划问题的解. 然后, 基于动态规划问题的解, 利用在线数据为静态优化问题提供了最优解. 最后, 仿真结果验证了该方法的有效性.     

The equivalent discrete-time optimal control problem for time-varying continuous-time systems with white stochastic parameters

The transformation into discrete-time equivalents of digital optimal control problems, involving continuous-time linear systems with white stochastic parameters, and quadratic integral criteria, is considered. The system parameters have time-varying statistics. The observations available at the sampling instants are in general nonlinear and corrupted by discrete-time noise. The equivalent discrete-time system has white stochastic parameters. Expressions are derived for the first and second moment of these parameters and for the parameters of the equivalent discrete-time sum criterion, which are explicit in the parameters and statistics of the original digital optimal control problem. A numerical algorithm to compute these expressions is presented. For each sampling interval, the algorithm computes the expressions recursively, forward in time, using successive equidistant evaluations of the matrices which determine the original digital optimal control problem. The algorithm is illustrated with three examples. If the observations at the sampling instants are linear and corrupted by multiplicative and/or additive discrete-time white noise, then, using recent results, full and reduced-order controllers that solve the equivalent discrete-time optimal control problem can be computed.     

Finite horizon quadratic optimal control and a separation principle for Markovian jump linear systems

In this note, we consider the finite-horizon quadratic optimal control problem of discrete-time Markovian jump linear systems driven by a wide sense white noise sequence. We assume that the output variable and the jump parameters are available to the controller. It is desired to design a dynamic Markovian jump controller such that the closed-loop system minimizes the quadratic functional cost of the system over a finite horizon period of time. As in the case with no jumps, we show that an optimal controller can be obtained from two coupled Riccati difference equations, one associated to the optimal control problem when the state variable is available, and the other one associated to the optimal filtering problem. This is a principle of separation for the finite horizon quadratic optimal control problem for discrete-time Markovian jump linear systems. When there is only one mode of operation our results coincide with the traditional separation principle for the linear quadratic Gaussian control of discrete-time linear systems.     

基于多参数规划的显式模型预测系统设计的可行域扩张算法

基于约束线性优化控制问题的多参数二次规划求解方法, 提出设计显式模型预测控制系统的可行域逐步扩张算法. 首先建立一种求取优化控制问题输出不变集的迭代算法. 以该输出不变集作为多参数规划问题中状态区域约束限制的初始条件, 通过反复求解多参数规划问题和不断改变状态区域约束限制, 能够逐步扩大显式模型预测控制系统的无限时间可行区域, 直到可行域不再继续扩大. 算法收敛时设计得到的显式模型预测控制系统在其所有的状态分区上都是无限时间可行的. 通过数值仿真计算, 验证本文提出算法的有效性.     

离散时间非线性时滞系统最优控制的DISOPE算法

对于非线性时滞系统的最优控制,提出一种基于线性时滞模型和二次型性能指标问题的迭代处蒙混过关针时滞系统化为满足可尔可夫性质的增广状态系统,在模型和实际存在差异的情况下,该算法通过迭代求解时滞线性最优控制问题和参数估计问题,获得原问题的最优解,仿真实例表明该算法的有效性和实用性。     

Discrete-time optimal fuzzy controller design: global conceptapproach

Proposes a systematic and theoretically sound way to design a global optimal discrete-time fuzzy controller to control and stabilize a nonlinear discrete-time fuzzy system with finite or infinite horizon (time). A linear-like global system representation of a discrete-time fuzzy system is first proposed by viewing such a system in a global concept and unifying the individual matrices into synthetic matrices. Then, based on this kind of system representation, a discrete-time optimal fuzzy control law which can achieve a global minimum effect is developed theoretically. A nonlinear two-point boundary-value-problem (TPBVP) is derived as a necessary and sufficient condition for the nonlinear quadratic optimal control problem. To simplify the computation, a multi-stage decomposition of the optimization scheme is proposed, and then a segmental recursive Riccati-like equation is derived. Moreover, in the case of time-invariant fuzzy systems, we show that the optimal controller can be obtained by just solving discrete-time algebraic Riccati-like equations. Based on this, several fascinating characteristics of the resultant closed-loop fuzzy system can easily be elicited. The stability of the closed-loop fuzzy system can be ensured by the designed optimal fuzzy controller. The optimal closed-loop fuzzy system can not only be guaranteed to be exponentially stable, but also stabilized to any desired degree. Also, the total energy of system output is absolutely finite. Moreover, the resultant closed-loop fuzzy system possesses an infinite gain margin, i.e. its stability is guaranteed no matter how large the feedback gain becomes. An example is given to illustrate the proposed optimal fuzzy controller design approach and to demonstrate the proven stability properties     

Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints

In this paper, the near-optimal control problem for a class of nonlinear discrete-time systems with control constraints is solved by iterative adaptive dynamic programming algorithm. First, a novel nonquadratic performance functional is introduced to overcome the control constraints, and then an iterative adaptive dynamic programming algorithm is developed to solve the optimal feedback control problem of the original constrained system with convergence analysis. In the present control scheme, there are three neural networks used as parametric structures for facilitating the implementation of the iterative algorithm. Two examples are given to demonstrate the convergence and feasibility of the proposed optimal control scheme.     

Optimal control of discrete-time linear fractional-order systems with multiplicative noise

A finite horizon linear quadratic (LQ) optimal control problem is studied for a class of discrete-time linear fractional systems (LFSs) affected by multiplicative, independent random perturbations. Based on the dynamic programming technique, two methods are proposed for solving this problem. The first one seems to be new and uses a linear, expanded-state model of the LFS. The LQ optimal control problem reduces to a similar one for stochastic linear systems and the solution is obtained by solving Riccati equations. The second method appeals to the principle of optimality and provides an algorithm for the computation of the optimal control and cost by using directly the fractional system. As expected, in both cases, the optimal control is a linear function in the state and can be computed by a computer program. A numerical example and comparative simulations of the optimal trajectory prove the effectiveness of the two methods. Some other simulations are obtained for different values of the fractional order.     

Bilinear black-box identification and MPC of the activated sludge process

In this paper the activated sludge process, which is a process for biological nitrogen removal in municipal wastewater treatment plants, is modeled as a discrete-time bilinear system by application of a recursive prediction error method system identification technique. A novel bilinear model predictive control algorithm is also derived and applied on a simulation model of the activated sludge process. For discrete-time bilinear systems, a quadratic cost on the predicted outputs and inputs, together with input/state constraints, results in a nonlinear non-convex optimization problem. An investigation is performed where the suggested control algorithm is compared with a linear counterpart. The results reveals that even though the identified bilinear black-box model describes the dynamics of the activated sludge process better than linear black-box models, bilinear model predictive control only gives moderate improvements of the control performance compared to linear model predictive control laws.