Optimal controller for uncertain stochastic polynomial systems with deterministic disturbances

This article presents the optimal quadratic-Gaussian controller for uncertain stochastic polynomial systems with unknown coefficients and matched deterministic disturbances over linear observations and a quadratic criterion. The optimal closed-form controller equations are obtained through the separation principle, whose applicability to the considered problem is substantiated. As intermediate results, this article gives closed-form solutions of the optimal regulator, controller and identifier problems for stochastic polynomial systems with linear control input and a quadratic criterion. The original problem for uncertain stochastic polynomial systems with matched deterministic disturbances is solved using the integral sliding mode algorithm. Performance of the obtained optimal controller is verified in the illustrative example against the conventional quadratic-Gaussian controller that is optimal for stochastic polynomial systems with known parameters and without deterministic disturbances. Simulation graphs demonstrating overall performance and computational accuracy of the designed optimal controller are included.     

Discrete-time optimal control for stochastic nonlinear polynomial systems

This paper presents a solution to the discrete-time optimal control problem for stochastic nonlinear polynomial systems over linear observations and a quadratic criterion. The solution is obtained in two steps: the optimal control algorithm is developed for nonlinear polynomial systems by considering complete information when generating a control law. Then, the state estimate equations for discrete-time stochastic nonlinear polynomial system over linear observations are employed. The closed-form solution is finally obtained substituting the state estimates into the obtained control law. The designed optimal control algorithm can be applied to both distributed and lumped systems. To show effectiveness of the proposed controller, an illustrative example is presented for a second degree polynomial system. The obtained results are compared to the optimal control for the linearized system.     

无限时间长时延网络控制系统的随机最优控制

考虑二次性能指标下线性网络控制系统的随机最优控制问题,建立了控制器为事件驱动时长时延线性网络控制系统的数学模型,证明了在无限时间情况下离散随机黎卡提代数方程解的存在性,设计出无限时间情况下线性网络控制系统的随机最优控制器,得到相应的最优性能指标的表达形式,并证明了相应的随机最优控制器可使网络控制系统均方指数稳定.最后以网络控制下的倒立摆为对象进行仿真研究,仿真结果表明该方法的正确性和有效性.     

Coherent quantum LQG control

Based on a recently developed notion of physical realizability for quantum linear stochastic systems, we formulate a quantum LQG optimal control problem for quantum linear stochastic systems where the controller itself may also be a quantum system and the plant output signal can be fully quantum. Such a control scheme is often referred to in the quantum control literature as “coherent feedback control”. It distinguishes the present work from previous works on the quantum LQG problem where measurement is performed on the plant and the measurement signals are used as the input to a fully classical controller with no quantum degrees of freedom. The difference in our formulation is the presence of additional non-linear and linear constraints on the coefficients of the sought after controller, rendering the problem as a type of constrained controller design problem. Due to the presence of these constraints, our problem is inherently computationally hard and this also distinguishes it in an important way from the standard LQG problem. We propose a numerical procedure for solving this problem based on an alternating projections algorithm and, as an initial demonstration of the feasibility of this approach, we provide fully quantum controller design examples in which numerical solutions to the problem were successfully obtained. For comparison, we also consider the case of classical linear controllers that use direct or indirect measurements, and show that there exists a fully quantum linear controller which offers an improvement in performance over the classical ones.     

An integrated optimal control algorithm for discrete-time nonlinear stochastic system

Consider a discrete-time nonlinear system with random disturbances appearing in the real plant and the output channel where the randomly perturbed output is measurable. An iterative procedure based on the linear quadratic Gaussian optimal control model is developed for solving the optimal control of this stochastic system. The optimal state estimate provided by Kalman filtering theory and the optimal control law obtained from the linear quadratic regulator problem are then integrated into the dynamic integrated system optimisation and parameter estimation algorithm. The iterative solutions of the optimal control problem for the model obtained converge to the solution of the original optimal control problem of the discrete-time nonlinear system, despite model-reality differences, when the convergence is achieved. An illustrative example is solved using the method proposed. The results obtained show the effectiveness of the algorithm proposed.     

Optimal preview control for a linear continuous-time stochastic control system in finite-time horizon

This paper discusses the design of the optimal preview controller for a linear continuous-time stochastic control system in finite-time horizon, using the method of augmented error system. First, an assistant system is introduced for state shifting. Then, in order to overcome the difficulty of the state equation of the stochastic control system being unable to be differentiated because of Brownian motion, the integrator is introduced. Thus, the augmented error system which contains the integrator vector, control input, reference signal, error vector and state of the system is reconstructed. This leads to the tracking problem of the optimal preview control of the linear stochastic control system being transformed into the optimal output tracking problem of the augmented error system. With the method of dynamic programming in the theory of stochastic control, the optimal controller with previewable signals of the augmented error system being equal to the controller of the original system is obtained. Finally, numerical simulations show the effectiveness of the controller.     

线性离散奇异系统的不定二次最优控制问题: Krein空间方法

The finite time horizon indefinite linear quadratic(LQ) optimal control problem for singular linear discrete time-varying systems is discussed. Indefinite LQ optimal control problem for singular systems can be transformed to that for standard state-space systems under a reasonable assumption. It is shown that the indefinite LQ optimal control problem is dual to that of projection for backward stochastic systems. Thus, the optimal LQ controller can be obtained by computing the gain matrices of Kalman filter. Necessary and sufficient conditions guaranteeing a unique solution for the indefinite LQ problem are given. An explicit solution for the problem is obtained in terms of the solution of Riccati difference equations.     

Preview control for a class of linear stochastic systems with multiplicative noise

ABSTRACT

In this paper, the preview control problem for a class of linear continuous time stochastic systems with multiplicative noise is studied based on the augmented error system method. First, a deterministic assistant system is introduced, and the original system is translated to the assistant system. Then, the integrator is employed to ensure the output of the closed-loop system tracking the reference signal accurately. Second, the augmented error system, which includes integrator vector, control vector and reference signal, is constructed based on the system after translation. As a result, the tracking problem is transformed into the optimal control problem of the augmented error system, and the optimal control input is obtained by the dynamic programming method. This control input is regarded as the preview controller of the original system. For a linear stochastic system with multiplicative noise, the difficulty being unable to construct an augmented error system by the derivation method is solved in this paper. And, the existence and uniqueness solution of the Riccati equation corresponding to the stochastic augmented error system is discussed. The numerical simulations show that the preview controller designed in this paper is very effective.     

输出概率密度函数鲁棒弹性最优跟踪控制

研究了一类随机动态系统的鲁棒弹性最优跟踪控制问题。在采用B样条神经网络模型逼近随机动态系统的输出概率密度函数(PDF)的基础上,同时考虑系统模型和控制器增益不确定性,结合Lyapunov稳定性理论和线性矩阵不等式(LMI)技术,引入增广控制作用,设计基于广义状态反馈的鲁棒弹性最优跟踪控制器,目的是使系统的输出PDF跟踪给定PDF。通过求解LMI,所得控制器不仅能实现跟踪目的,而且能确保该随机动态系统全局稳定并满足一定的线性二次型性能指标上界。仿真结果表明该方法简单易行,且无需任何设计参数调整。     

Robust tracking problem for continuous time stochastic uncertain systems

We propose a finite‐horizon robust minimax tracking controller design method for time‐varying continuous time stochastic uncertain systems. The uncertainty in the system is characterized by a set of probability measures under which stochastic noises, driving the system, are defined. A minimax optimal tracking controller is derived from the solution of a risk‐sensitive linear quadratic Gaussian control problem. Also a numerical example is presented to illustrate the characteristics of proposed tracking controller. Copyright © 2008 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Stochastic differential games and inverse optimal control and stopper policies

In this paper, we consider a two-player stochastic differential game problem over an infinite time horizon where the players invoke controller and stopper strategies on a nonlinear stochastic differential game problem driven by Brownian motion. The optimal strategies for the two players are given explicitly by exploiting connections between stochastic Lyapunov stability theory and stochastic Hamilton–Jacobi–Isaacs theory. In particular, we show that asymptotic stability in probability of the differential game problem is guaranteed by means of a Lyapunov function which can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton–Jacobi–Isaacs equation, and hence, guaranteeing both stochastic stability and optimality of the closed-loop control and stopper policies. In addition, we develop optimal feedback controller and stopper policies for affine nonlinear systems using an inverse optimality framework tailored to the stochastic differential game problem. These results are then used to provide extensions of the linear feedback controller and stopper policies obtained in the literature to nonlinear feedback controllers and stoppers that minimise and maximise general polynomial and multilinear performance criteria.     

Profitable and dynamically feasible operating point selection for constrained processes

The operating point of a typical chemical process is determined by solving a non-linear optimization problem where the objective is to minimize an economic cost subject to constraints. Often, some or all of the constraints at the optimal solution are active, i.e., the solution is constrained. Though it is profitable to operate at the constrained optimal point, it might lead to infeasible operation due to uncertainties. Hence, industries try to operate the plant close to the optimal point by “backing-off” to achieve the desired economic benefits. Therefore, the primary focus of this paper is to present an optimization formulation for solving the dynamic back-off problem based on an economic cost function. In this regard, we work within a stochastic framework that ensures feasible dynamic operating region within the prescribed confidence limit. In this work, we aim to reduce the economic loss due to the back-off by simultaneously solving for the operating point and a compatible controller that ensures feasibility. Since the resulting formulation is non-linear and non-convex, we propose a novel two-stage iterative solution procedure such that a convex problem is solved at each step in the iteration. Finally, the proposed approach is demonstrated using case studies.     

Exact optimal solution for a class of dual control problems

This paper considers a discrete-time stochastic optimal control problem for which only measurement equation is partially observed with unknown constant parameters taking value in a finite set of stochastic systems. Because of the fact that the cost-to-go function at each stage contains variance and the non-separability of the variance is so complicated that the dynamic programming cannot be successfully applied, the optimal solution has not been found. In this paper, a new approach to the optimal solution is proposed by embedding the original non-separable problem into a separable auxiliary problem. The theoretical condition on which the optimal solution of the original problem can be attained from a set of solutions of the auxiliary problem is established. In addition, the optimality of the interchanging algorithm is proved and the analytical solution of the optimal control is also obtained. The performance of this controller is illustrated with a simple example.     

带有随机跳跃干扰的线性二次随机最优控制问题

给出一类布朗运动和泊松过程混合驱动的正倒向随机微分方程解的存在唯一性结果, 应用这一结果研究带有随机跳跃干扰的线性二次随机最优控制问题,并得到最优控制的显式形 式,可以证明最优控制是唯一的.然后,引入和研究一类推广的黎卡提方程系统,讨论该方程系统 的可解性并由该方程的解得到带有随机跳跃干扰的线性二次随机最优控制问题最优的线性反馈.     

Nonlinear stochastic receding horizon control: stability,robustness and Monte Carlo methods for control approximation

This work considers the stability of nonlinear stochastic receding horizon control when the optimal controller is only computed approximately. A number of general classes of controller approximation error are analysed including deterministic and probabilistic errors and even controller sample and hold errors. In each case, it is shown that the controller approximation errors do not accumulate (even over an infinite time frame) and the process converges exponentially fast to a small neighbourhood of the origin. In addition to this analysis, an approximation method for receding horizon optimal control is proposed based on Monte Carlo simulation. This method is derived via the Feynman–Kac formula which gives a stochastic interpretation for the solution of a Hamilton–Jacobi–Bellman equation associated with the true optimal controller. It is shown, and it is a prime motivation for this study, that this particular controller approximation method practically stabilises the underlying nonlinear process.     

Adaptive control design under structured model information limitation: A cost-biased maximum-likelihood approach

Networked control strategies based on limited information about the plant model usually result in worse closed-loop performance than optimal centralized control with full plant model information. Recently, this fact has been established by utilizing the concept of competitive ratio, which is defined as the worst-case ratio of the cost of a control design with limited model information to the cost of the optimal control design with full model information. We show that an adaptive controller, inspired by a controller proposed by Campi and Kumar, with limited plant model information, asymptotically achieves the closed-loop performance of the optimal centralized controller with full model information for almost any plant. Therefore, there exists, at least, one adaptive control design strategy with limited plant model information that can achieve a competitive ratio equal to one. The plant model considered in the paper belongs to a compact set of stochastic linear time-invariant systems and the closed-loop performance measure is the ergodic mean of a quadratic function of the state and control input.     

An optimal control approach to the optical flow problem

The optical flow problem is reduced to an optimal control problem governed by a linear parabolic equation having the unknown velocity field (the optical flow) as drift term. This model is derived from a new assumption, that is, the brightness intensity is conserved on a moving pattern driven by a Gaussian stochastic process. The optimality conditions are deduced by a passage to the limit technique in an approximating optimal control problem introduced for a regularization purpose. Finally, the controller uniqueness is addressed.     

Optimal quadratic solution for the non-Gaussian finite-horizon regulator problem

In this paper, the problem of the optimal quadratic regulator for non-Gaussian discrete-time stochastic systems with a quadratic cost function is considered. The main result here obtained is that such optimal control can be derived from the classical LQG solution by substituting the linear filtering part with a quadratic optimal filter. Numerical results show high performance of this method.     

Discrete-time controller for stochastic nonlinear polynomial systems with Poisson noises

This paper presents a solution to the optimal control problem for discrete-time stochastic nonlinear polynomial systems confused with white Poisson noises subject to a quadratic criterion. The solution is obtained in the following way: a nonlinear optimal controller is first developed for polynomial systems, considering the state vector completely available for control design. Then, based on the solution of the state estimation problem for polynomial systems with white Poisson noises, the state estimate vector is used in the control law to obtain a closed-form solution. Performance of this controller is compared to that of the controller employing the extended Kalman filter and the linear-quadratic regulator and the controller designed for polynomial systems confused with white Gaussian noises.