Discrete-time optimal control for stochastic nonlinear polynomial systems

This paper presents a solution to the discrete-time optimal control problem for stochastic nonlinear polynomial systems over linear observations and a quadratic criterion. The solution is obtained in two steps: the optimal control algorithm is developed for nonlinear polynomial systems by considering complete information when generating a control law. Then, the state estimate equations for discrete-time stochastic nonlinear polynomial system over linear observations are employed. The closed-form solution is finally obtained substituting the state estimates into the obtained control law. The designed optimal control algorithm can be applied to both distributed and lumped systems. To show effectiveness of the proposed controller, an illustrative example is presented for a second degree polynomial system. The obtained results are compared to the optimal control for the linearized system.     

Mixed Constrained Infinite Horizon Linear Quadratic Optimal Control

For a given initial state, a constrained infinite horizon linear quadratic optimal control problem can be reduced to a finite dimensional problem [12]. To find a conservative estimate of the size of the reduced problem, the existing algorithms require the on‐line solutions of quadratic programs [10] or a linear program [2]. In this paper, we first show based on the Lyapunov theorem that the closed‐loop system with a mixed constrained infinite horizon linear quadratic optimal control is exponentially stable on proper sets. Then the exponentially converging envelop of the closed‐loop trajectory that can be computed off‐line is employed to obtain a finite dimensional quadratic program equivalent to the mixed constrained infinite horizon linear quadratic optimal control problem without any on‐line optimization. The example considered in [2] showed that the proposed algorithm identifies less conservative size estimate of the reduced problem with much less computation.     

Optimal controller for uncertain stochastic polynomial systems with deterministic disturbances

This article presents the optimal quadratic-Gaussian controller for uncertain stochastic polynomial systems with unknown coefficients and matched deterministic disturbances over linear observations and a quadratic criterion. The optimal closed-form controller equations are obtained through the separation principle, whose applicability to the considered problem is substantiated. As intermediate results, this article gives closed-form solutions of the optimal regulator, controller and identifier problems for stochastic polynomial systems with linear control input and a quadratic criterion. The original problem for uncertain stochastic polynomial systems with matched deterministic disturbances is solved using the integral sliding mode algorithm. Performance of the obtained optimal controller is verified in the illustrative example against the conventional quadratic-Gaussian controller that is optimal for stochastic polynomial systems with known parameters and without deterministic disturbances. Simulation graphs demonstrating overall performance and computational accuracy of the designed optimal controller are included.     

Fast algorithm for a constrained infinite horizon LQ problem

This paper presents an efficient algorithmic solution to the infinite horizon linear quadratic optimal control problem for a discrete-time SISO plant subject to bound constraints on a scalar variable. The solution to the corresponding quadratic programming problem is based on the active set method and on dynamic programming. It is shown that the optimal solution can be updated after inclusion or removal of an active constraint by a simple procedure requiring in the order of kn operations, n being the system order and k the time at which the constraint is included or removed.     

Adaptive dynamic programming for online solution of a zero-sum differential game

This paper will present an approximate/adaptive dynamic programming(ADP) algorithm,that uses the idea of integral reinforcement learning(IRL),to determine online the Nash equilibrium solution for the two-player zerosum differential game with linear dynamics and infinite horizon quadratic cost.The algorithm is built around an iterative method that has been developed in the control engineering community for solving the continuous-time game algebraic Riccati equation(CT-GARE),which underlies the game problem.We here show how the ADP techniques will enhance the capabilities of the offline method allowing an online solution without the requirement of complete knowledge of the system dynamics.The feasibility of the ADP scheme is demonstrated in simulation for a power system control application.The adaptation goal is the best control policy that will face in an optimal manner the highest load disturbance.     

Optimal sampling-rates and tracking properties of digital LQ and LQG tracking controllers for systems with an exogenous component and costs associated to sampling

In this paper it is shown that if costs are associated to sampling operations which are added to a performance criterion, the minimization of this new performance criterion results in a controller operated at an optimal sampling rate. This, under the assumptions that the system is periodically sampled, the applied control is kept fixed between every two sampling instances and some technical conditions are met. In case the considered planning horizon in the performance criterion is finite an algorithm is devised which calculates in a finite number of steps the optimal sampling period. It is shown that the technical conditions mentioned above are satisfied by the finite planning horizon time-varying LQG tracking problem. Since stability is a major requirement in controller design we also consider the case of an infinite planning horizon. This analysis is focused on the time-invariant digital LQ tracking problem. Given some mild regularity conditions a numerical algorithm is presented that approximates the optimal solution within any prespecified error norm. It is shown that also in this case an optimal sampling-rate exists. The algorithm for determining the optimal sampling period if the planning horizon is finite is illustrated in an economic example.     

A modified genetic algorithm for optimal control problems

This paper studies the application of a genetic algorithm to discrete-time optimal control problems. Numerical results obtained here are compared with ones yielded by GAMS, a system for construction and solution of large and complex mathematical programming models. While GAMS appears to work well only for linear quadratic optimal control problems or problems with short horizon, the genetic algorithm applies to more general problems equally well.     

Quadratic control of stochastic hybrid systems with renewal transitions

We study the quadratic control of a class of stochastic hybrid systems with linear continuous dynamics for which the lengths of time that the system stays in each mode are independent random variables with given probability distribution functions. We derive a condition for finding the optimal feedback policy that minimizes a discounted infinite horizon cost. We show that the optimal cost is the solution to a set of differential equations with unknown boundary conditions. Furthermore, we provide a recursive algorithm for computing the optimal cost and the optimal feedback policy. The applicability of our result is illustrated through a numerical example, motivated by stochastic gene regulation in biology.     

A fuzzy control system with application to production planning problems

A considerable part of the literature on fuzzy sets is devoted to the field of fuzzy control system. In this paper, an alternative control system is introduced to describe a dynamic system with fuzzy white noise. In order to find optimal ways to control such a system, fuzzy optimal control theory is further developed. Specifically, a linear quadratic model is formulated and solved as a fuzzy optimal control problem. The formulation and solution of this model provide an economic interpretation of a production planning model both in the finite horizon and in the infinite horizon.     

Discrete-time controller for stochastic nonlinear polynomial systems with Poisson noises

This paper presents a solution to the optimal control problem for discrete-time stochastic nonlinear polynomial systems confused with white Poisson noises subject to a quadratic criterion. The solution is obtained in the following way: a nonlinear optimal controller is first developed for polynomial systems, considering the state vector completely available for control design. Then, based on the solution of the state estimation problem for polynomial systems with white Poisson noises, the state estimate vector is used in the control law to obtain a closed-form solution. Performance of this controller is compared to that of the controller employing the extended Kalman filter and the linear-quadratic regulator and the controller designed for polynomial systems confused with white Gaussian noises.     

A rigorous solution of the infinite time interval LQ problem with constant state tracking

A linear quadratic constant state tracking problem is considered over an infinite time horizon. It is shown that the solution cannot be obtained as a limit from a finite time horizon problem, as in general the limiting problem is ill-posed. To obtain a rigorous solution, the problem is split in two natural well-posed subproblems. One optimal control problem addresses the transient and the other optimal control problem concerns the steady state behavior. It is shown that the transient problem and the steady state problem are solved by the same control law.     

Infinite horizon linear quadratic differential games for discrete-time stochastic systems

This paper deals with the infinite horizon linear quadratic(LQ)differential games for discrete-time stochastic systems with both state and control dependent noise.The Popov-Belevitch-Hautus(PBH)criteria for exact observability and exact detectability of discrete-time stochastic systems are presented.By means of them,we give the optimal strategies (Nash equilibrium strategies)and the optimal cost values for infinite horizon stochastic differential games.It indicates that the infinite horizon LQ stochastic differential games are associated with four coupled matrix-valued equations.Furthermore, an iterative algorithm is proposed to solve the four coupled equations.Finally,an example is given to demonstrate our results.     

无限时间长时延网络控制系统的随机最优控制

考虑二次性能指标下线性网络控制系统的随机最优控制问题,建立了控制器为事件驱动时长时延线性网络控制系统的数学模型,证明了在无限时间情况下离散随机黎卡提代数方程解的存在性,设计出无限时间情况下线性网络控制系统的随机最优控制器,得到相应的最优性能指标的表达形式,并证明了相应的随机最优控制器可使网络控制系统均方指数稳定.最后以网络控制下的倒立摆为对象进行仿真研究,仿真结果表明该方法的正确性和有效性.     

Adaptive optimal control for continuous-time linear systems based on policy iteration

In this paper we propose a new scheme based on adaptive critics for finding online the state feedback, infinite horizon, optimal control solution of linear continuous-time systems using only partial knowledge regarding the system dynamics. In other words, the algorithm solves online an algebraic Riccati equation without knowing the internal dynamics model of the system. Being based on a policy iteration technique, the algorithm alternates between the policy evaluation and policy update steps until an update of the control policy will no longer improve the system performance. The result is a direct adaptive control algorithm which converges to the optimal control solution without using an explicit, a priori obtained, model of the system internal dynamics. The effectiveness of the algorithm is shown while finding the optimal-load-frequency controller for a power system.     

Infinite horizon optimal control of linear discrete time systems with stochastic parameters

The infinite horizon optimal control problem is considered in the general case of linear discrete time systems and quadratic criteria, both with stochastic parameters which are independent with respect to time. A stronger stabilizability property and a weaker observability property than usual for deterministic systems are introduced. It is shown that the infinite horizon problem has a solution if the system has the first property. If in addition the problem has the second property the solution is unique and the control system is stable in the mean square sense. A simple necessary and sufficient condition, explicit in the system matrices, is given for the system to have the stronger stabilizability property. This condition also holds for deterministic systems to be stabilizable in the usual sense. The stronger stabilizability and weaker observability properties coincide with the usual ones if the parameters are deterministic.     

Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics

This paper develops an online algorithm based on policy iteration for optimal control with infinite horizon cost for continuous-time nonlinear systems. In the present method, a discounted value function is employed, which is considered to be a more general case for optimal control problems. Meanwhile, without knowledge of the internal system dynamics, the algorithm can converge uniformly online to the optimal control, which is the solution of the modified Hamilton–Jacobi–Bellman equation. By means of two neural networks, the algorithm is able to find suitable approximations of both the optimal control and the optimal cost. The uniform convergence to the optimal control is shown, guaranteeing the stability of the nonlinear system. A simulation example is provided to illustrate the effectiveness and applicability of the present approach.     

Control of power systems by constrained tracking methods

The problem of improving the dynamic response of a power system is considered. It is formulated as a linear optimal control problem with a quadratic tracking criterion and control constraints. A computational algorithm is developed. Simulation studies are performed on a system consisting of a synchronous machine connected to a load through an infinite bus using governor and excitation controls.     

基于动态规划的约束优化问题多参数规划求解方法及应用

结合动态规划和单步多参数二次规划, 提出一种新的约束优化控制问题多参数规划求解方法. 一方面能得到约束线性二次优化控制问题最优控制序列与状态之间的显式函数关系, 减少多参数规划问题求解的工作量; 另一方面能够同时求解得到状态反馈最优控制律. 应用本文提出的多参数二次规划求解方法, 建立无限时间约束优化问题状态反馈显式最优控制律. 针对电梯机械系统振动控制模型做了数值仿真计算.     

Guaranteeing cost strategies for linear quadratic differential games under uncertain dynamics

This paper deals with the design of closed loop strategies for a class of two players zero-sum linear quadratic differential games, where each player does not know exactly the state equation and model it through a system subject to norm-bounded uncertainties. The finite horizon and the infinite horizon problems are both solved: it turns out that the optimal strategies, guaranteeing to each player a given level of performance, require, to be evaluated, the solution of two scaled differential (algebraic in the infinite horizon case) Riccati equations. A numerical example illustrates an application of the proposed technique.