不确定拟哈密顿系统的随机最优控制

本文提出了不确定拟哈密顿系统、基于随机平均法、随机极大值原理和随机微分对策理论的一种随机极大极小最优控制策略.首先,运用拟哈密顿系统的随机平均法,将系统状态从速度和位移的快变量形式转化为能量的慢变量形式,得到部分平均的It随机微分方程;其次,给定控制性能指标,对于不确定拟哈密顿系统的随机最优控制,根据随机微分对策理论,将其转化为一个极小极大控制问题;再根据随机极大值原理,建立关于系统与伴随过程的前向-后向随机微分方程,随机最优控制表达为哈密顿控制函数的极大极小条件,由此得到最坏情形下的扰动参数与极大极小最优控制;然后,将最坏扰动参数与最优控制代入部分平均的It随机微分方程并完成平均,求解与完全平均的It随机微分方程相应的Fokker-Planck-Kolmogorov(FPK)方程,可得受控系统的响应量并计算控制效果;最后,将上述不确定拟哈密顿系统的随机最优控制策略应用于一个两自由度非线性系统,通过数值结果说明该随机极大极小控制策略的控制效果.     

Stochastic minimax optimal control strategy for uncertain quasi-Hamiltonian systems using stochastic maximum principle

A stochastic minimax optimal control strategy for uncertain quasi-Hamiltonian systems is proposed based on the stochastic averaging method, stochastic maximum principle and stochastic differential game theory. First, the partially completed averaged Itô stochastic differential equations are derived from a given system by using the stochastic averaging method for quasi-Hamiltonian systems with uncertain parameters. Then, the stochastic Hamiltonian system for minimax optimal control with a given performance index is established based on the stochastic maximum principle. The worst disturbances are determined by minimizing the Hamiltonian function, and the worst-case optimal controls are obtained by maximizing the minimal Hamiltonian function. The differential equation for adjoint process as a function of system energy is derived from the adjoint equation by using the Itô differential rule. Finally, two examples of controlled uncertain quasi-Hamiltonian systems are worked out to illustrate the application and effectiveness of the proposed control strategy.     

Robust filtering of stochastic uncertain systems on an infinite time horizon

In this paper, we consider a robust filtering problem for continuous time stochastic uncertain systems.The uncertainty in the system is characterized in terms of an uncertain probability distribution on the noise input. This uncertainty is assumed to satisfy a certain relative entropy constraint. The solution to a specially parameterized risk-sensitive stochastic filtering problem is used to construct a filter for the original uncertain system which guarantees an optimal worst-case filtering error. The corresponding minimax optimal filter is obtained by solving a pair of algebraic Riccati equations.     

Minimax games for stochastic systems subject to relative entropy uncertainty: applications to SDEs on Hilbert spaces

In this paper, we consider minimax games for stochastic uncertain systems with the pay-off being a nonlinear functional of the uncertain measure where the uncertainty is measured in terms of relative entropy between the uncertain and the nominal measure. The maximizing player is the uncertain measure, while the minimizer is the control which induces a nominal measure. Existence and uniqueness of minimax solutions are derived on suitable spaces of measures. Several examples are presented illustrating the results. Subsequently, the results are also applied to controlled stochastic differential equations on Hilbert spaces. Based on infinite dimensional extension of Girsanov’s measure transformation, martingale solutions are used in establishing existence and uniqueness of minimax strategies. Moreover, some basic properties of the relative entropy of measures on infinite dimensional spaces are presented and then applied to uncertain systems described by a stochastic differential inclusion on Hilbert space. An explicit expression for the worst case measure representing the maximizing player (adversary) is found.     

Finite Horizon Minimax Optimal Control of Stochastic Partially Observed Time Varying Uncertain Systems

We consider a linear-quadratic problem of minimax optimal control for stochastic uncertain control systems with output measurement. The uncertainty in the system satisfies a stochastic integral quadratic constraint. To convert the constrained optimization problem into an unconstrained one, a special S-procedure is applied. The resulting unconstrained game-type optimization problem is then converted into a risk-sensitive stochastic control problem with an exponential-of-integral cost functional. This is achieved via a certain duality relation between stochastic dynamic games and risk-sensitive stochastic control. The solution of the risk-sensitive stochastic control problem in terms of a pair of differential matrix Riccati equations is then used to establish a minimax optimal control law for the original uncertain system with uncertainty subject to the stochastic integral quadratic constraint. Date received: May 13, 1997. Date revised: March 18, 1998.     

Robust tracking problem for continuous time stochastic uncertain systems

We propose a finite‐horizon robust minimax tracking controller design method for time‐varying continuous time stochastic uncertain systems. The uncertainty in the system is characterized by a set of probability measures under which stochastic noises, driving the system, are defined. A minimax optimal tracking controller is derived from the solution of a risk‐sensitive linear quadratic Gaussian control problem. Also a numerical example is presented to illustrate the characteristics of proposed tracking controller. Copyright © 2008 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games

Entropy and relative entropy are fundamental concepts on which information theory is founded on, and in general, telecommunication systems design. On the other hand, dissipation inequalities, minimax strategies, and induced norms are the basic concepts on which robustness of uncertain control and estimation of systems are founded on. In this paper, the precise relation between these notions is investigated. In particular, it will be shown that the higher the dissipation the higher the entropy of the system, which has implications in computing the induced norm associated with robustness. These connections are obtained by considering stochastic optimal uncertain control systems, in which uncertainty is described by a relative entropy constraint between the nominal and uncertain measures, while the pay-off is a linear functional of the uncertain measure. This is a minimax game, in which the controller measure seeks to minimize the pay-off, while the disturbance measure aims at maximizing the pay-off. Salient properties of the minimax solution are derived, including a characterization of the optimal sensitivity reduction, computation of the induced norm, monotonicity properties of minimax solution, and relations between dissipation and relative entropy of the system. The theory is developed in an abstract setting and then applied to nonlinear partially observable continuous-time uncertain controlled systems, in which the nominal and uncertain systems are described by conditional distributions. In addition, existence of the optimal control policy among the class of policies known as wide-sense control laws is shown, and an explicit formulae for the worst case conditional measure is derived. The results are applied to linear-quadratic-Gaussian problems     

Guaranteed cost LQG control for uncertain systems with a normalized coprime factor uncertainty structure

The paper extends the robust minimax LQG control design methodology to stochastic uncertain systems with a general uncertainty structure, which includes normalized coprime factor uncertainty, passive uncertainty and sector norm-bounded uncertainty as special cases. The derivation of the result uses a special parameter-dependent Girsanov measure transformation. The efficacy of the proposed methodology is illustrated using the problem of frequency locking of an optical cavity which occurs in the area of experimental quantum optics.     

LMI-based minimax estimation and filtering under unknown covariances

In this paper, we consider a minimax approach to the estimation and filtering problems in the stochastic framework, where covariances of the random factors are completely unknown. The term ‘random factors’ refers either to unknown parameters and measurement noise in the estimation problem or to disturbance process and the initial state of a linear discrete-time dynamic system in the filtering problem. We introduce a notion of the attenuation level of random factors as a performance measure for both a linear unbiased estimate and a filter. This is the worst-case variance of the estimation error normalised by the sum of variances of all random factors over all nonzero covariance matrices. It is shown that this performance measure is equal to the spectral norm of the ‘transfer matrix’ and therefore the minimax estimate and filter can be computed in terms of linear matrix inequalities (LMIs). Moreover, the explicit formulae for both the minimax estimate and the minimal value of the attenuation level are presented in the estimation problem. It turns out that the above attenuation level of random factors coincides with the attenuation level of deterministic factors that is the worst-case normalised squared Euclidian norm of the estimation error over all nonzero sample values of random factors. In addition, we demonstrate that the LMI technique can be applied to derive the optimal robust estimator and filter, when there is a priori information about convex polyhedral sets which unknown covariance matrices of random factors belong to. Two illustrative examples show advantages of the minimax approach proposed.     

Robust finite horizon minimax filtering for discrete-time stochastic uncertain systems

We study a finite-horizon robust minimax filtering problem for time-varying discrete-time stochastic uncertain systems. The uncertainty in the system is characterized by a set of probability measures under which the stochastic noises, driving the system, are defined. The optimal minimax filter has been found by applying techniques of risk-sensitive LQG control. The structure and properties of resulting filter are analyzed and compared to H_∞ and Kalman filters.     

Finite Horizon Minimax Optimal Control of Nonlinear Continuous Time Systems with Stochastic Uncertainty

This paper develops a general continuous-time stochastic framework for robustness analysis and robust control synthesis. We consider a stochastic minimax optimization problem for general stochastic uncertain systems. A general method is presented for converting problems of performance analysis or controller synthesis into unconstrained optimization problems.     

Robust filtering for a class of nonlinear stochastic systems with probability constraints

This paper is concerned with the probability-constrained filtering problem for a class of time-varying nonlinear stochastic systems with estimation error variance constraint. The stochastic nonlinearity considered is quite general that is capable of describing several well-studied stochastic nonlinear systems. The second-order statistics of the noise sequence are unknown but belong to certain known convex set. The purpose of this paper is to design a filter guaranteeing a minimized upper-bound on the estimation error variance. The existence condition for the desired filter is established, in terms of the feasibility of a set of difference Riccati-like equations, which can be solved forward in time. Then, under the probability constraints, a minimax estimation problem is proposed for determining the suboptimal filter structure that minimizes the worst-case performance on the estimation error variance with respect to the uncertain second-order statistics. Finally, a numerical example is presented to show the effectiveness and applicability of the proposed method.     

Gain-Scheduling of Minimax Optimal State-Feedback Controllers for Uncertain LPV Systems

A new robust control gain-scheduling scheme for uncertain linear parameter-varying (LPV) systems is proposed. The gain-scheduled controller consists of a set of minimax optimal robust controllers and incorporates a new interpolation rule to achieve continuity of the controller gain over a range of operating conditions. For every fixed system parameter, the proposed controller guarantees a certain bound on the worst-case performance of the corresponding uncertain closed loop system. Furthermore, a bound on the rate of parameter variations is obtained under which the closed loop LPV system is robustly stable     

带不确定参数和噪声方差的鲁棒观测融合Kalman滤波器

对带不确定参数和噪声方差的多传感器定常系统,引入虚拟白噪声补偿不确定参数,可将其转化为带已知参数和不确定噪声方差系统.应用极大极小鲁棒估值原理和加权最小二乘法,基于带噪声方差保守上界的最坏情形保守系统,提出了鲁棒加权观测融合Kalman滤波器,并证明了它与集中式融合鲁棒Kalman滤波器是等价的,且融合器的鲁棒精度高于每个局部滤波器鲁棒精度.一个Monte-Carlo仿真例子说明了如何寻求不确定参数的鲁棒域和如何搜索保守性较小的虚拟噪声方差上界.     

State estimation with probability constraints

This paper considers a state estimation problem for a discrete-time linear system driven by a Gaussian random process. The second order statistics of the input process and state initial condition are uncertain. However, the probability that the state and input satisfy linear constraints during the estimation interval is known. A minimax estimation problem is formulated to determine an estimator that minimises the worst-case mean square error criterion, over the uncertain second order statistics, subject to the probability constraints. It is shown that a solution to this constrained state estimation problem is given by a Kalman filter for appropriately chosen input and initial condition models. These models are obtained from a finite dimensional convex optimisation problem. The application of this estimator to an aircraft tracking problem quantifies the improvement in estimation accuracy obtained from the inclusion of probability constraints in the minimax formulation.     

Minimax optimal control of uncertain systems with structured uncertainty

This paper considers a minimax control problem for an uncertain system containing structured uncertainties. The uncertainties in this system are assumed to satisfy a certain integral quadratic constraint. For a given initial condition, the minimax optimal controller is constructed by solving a parameter-dependent Riccati equation of the game type. This controller leads to a closed-loop uncertain system which is absolutely stable.     

具有噪声方差及多种网络诱导不确定系统鲁棒Kalman估计

对不确定噪声方差乘性噪声,同时带观测缺失、丢包和一步随机观测滞后三种网络诱导特征的混合不确定网络化系统,应用带虚拟噪声的扩维方法和去随机参数方法,将其转化为带不确定虚拟噪声方差的时变系统.基于极大极小鲁棒估计原理,对带虚拟噪声方差保守上界的最坏情形系统,设计了鲁棒时变和稳态Kalman估值器.对所有容许的不确定性,保证实际Kalman估计误差方差有最小上界.应用扩展的Lyapunov方程方法和矩阵分解方法证明了所设计估值器的鲁棒性.证明了实际和保守估值器的精度关系,以及时变和稳态估值器间的按实现收敛性.应用于F-404航空发动机系统的仿真验证了所提出结果的正确性和有效性.     

Design of minimax robust LQG controllers under parameter and noise uncertainties

The problem of design of minimax robust LQG controllers for linear systems with parameter and noise uncertainties is considered in this paper. Necessary and sufficient conditions for converting this problem to a two-person, zero-sum continuous game problem are presented. A simple procedure for design of a suboptimal minimax robust LQG controller, i.e., the LQG controller for least-favourable model, is proposed. Necessary and sufficient conditions for the existence of a saddle point are established. Under these conditions, the controller obtained is exactly the minimax LQG controller. When there does not exist a saddle point, the worst-case error between the controller obtained and the minimax robust LQG controllers under described uncertainties is bounded.     

Parallel design of robust control in the stochastic environment (the two-armed bandit problem)

The problem of rational behavior in the stochastic environment, also known as the two armed bandit problem, is considered in the robust (minimax) setting. A parallel strategy is proposed leading to control, which is arbitrary close to the optimal one for environments with gains having gaussian cumulative distribution functions with unit variance. The invariant recursive equation is obtained for computing the minimax strategy and risk, which are to be found as Bayesian ones associated with the worst-case a priori distribution. As a result, the well-known Vogel’s estimates of the minimax risk can be improved. Numerical experiments show that the strategy is efficient in the environments with non-gaussian distributions, e.g., the binary ones.     

Robust Control of Nonlinear Jump Parameter Systems Governed by Uncertain Chains

We consider an infinite-horizon minimax optimal control problem for stochastic uncertain systems governed by a discrete-state uncertain continuous-time chain. Using existing risk-sensitive control results, a robust suboptimal absolutely stabilizing guaranteed cost controller is constructed. Conditions are presented under which this suboptimal controller is minimax optimal. We then present a numeric algorithm for calculating a robust (sub)optimal controller using a Markov chain approximation technique.