不确定拟哈密顿系统的随机最优控制

本文提出了不确定拟哈密顿系统、基于随机平均法、随机极大值原理和随机微分对策理论的一种随机极大极小最优控制策略.首先,运用拟哈密顿系统的随机平均法,将系统状态从速度和位移的快变量形式转化为能量的慢变量形式,得到部分平均的It随机微分方程;其次,给定控制性能指标,对于不确定拟哈密顿系统的随机最优控制,根据随机微分对策理论,将其转化为一个极小极大控制问题;再根据随机极大值原理,建立关于系统与伴随过程的前向-后向随机微分方程,随机最优控制表达为哈密顿控制函数的极大极小条件,由此得到最坏情形下的扰动参数与极大极小最优控制;然后,将最坏扰动参数与最优控制代入部分平均的It随机微分方程并完成平均,求解与完全平均的It随机微分方程相应的Fokker-Planck-Kolmogorov(FPK)方程,可得受控系统的响应量并计算控制效果;最后,将上述不确定拟哈密顿系统的随机最优控制策略应用于一个两自由度非线性系统,通过数值结果说明该随机极大极小控制策略的控制效果.     

Stochastic maximum principle for optimal control problems involving delayed systems

Dear editor, The main objective of this study is to investigate one type of stochastic optimal control problem for a delayed system using the maximum principle ...     

Stochastic maximum principle for delayed doubly stochastic control systems and their applications

ABSTRACT

In this paper, we investigate the optimal control problems for delayed doubly stochastic control systems. We first discuss the existence and uniqueness of the delayed doubly stochastic differential equation by martingale representation theorem and contraction mapping principle. As a necessary condition of the optimal control, we deduce a stochastic maximum principle under some assumption. At the same time, a sufficient condition of optimality is obtained by using the duality method. At the end of the paper, we apply our stochastic maximum principle to a class of linear quadratic optimal control problem and obtain the explicit expression of the optimal control.     

Stochastic minimax control for stabilizing uncertain quasi- integrable Hamiltonian systems

A procedure for designing feedback control to asymptotically stabilize, with probability one, quasi-integrable Hamiltonian systems with bounded uncertain parametric disturbances is proposed. First, the partially averaged Itô stochastic differential equations are derived from given system by using the stochastic averaging method for quasi-integrable Hamiltonian systems. Second, the Hamilton-Jacobi-Issacs (HJI) equation for the ergodic control problem of the averaged system and a performance index with undetermined cost function is established based on the principle of optimality. This equation is then solved to yield the worst disturbances and the associated optimal controls. Third, the asymptotic Lyapunov stability with probability one of the optimally controlled system with worst disturbances is analyzed by evaluating the maximal Lyapunov exponent of the fully averaged Itô equations. Finally, the cost function and feedback control are determined by the requirement of stabilizing the worst-disturbed system. A simple example is worked out to illustrate the application of the proposed procedure and the effects of optimal control on stabilizing the uncertain system.     

带Markov跳的离散时间随机控制系统的最大值原理

本文研究一类同时含有Markov跳过程和乘性噪声的离散时间非线性随机系统的最优控制问题, 给出并证明了相应的最大值原理. 首先, 利用条件期望的平滑性, 通过引入具有适应解的倒向随机差分方程, 给出了带有线性差分方程约束的线性泛函的表示形式, 并利用Riesz定理证明其唯一性. 其次, 对带Markov跳的非线性随机控制系统, 利用针状变分法, 对状态方程进行一阶变分, 获得其变分所满足的线性差分方程. 然后, 在引入Hamilton函数的基础上, 通过一对由倒向随机差分方程刻画的伴随方程, 给出并证明了带有Markov跳的离散时间非线性随机最优控制问题的最大值原理, 并给出该最优控制问题的一个充分条件和相应的Hamilton-Jacobi-Bellman方程. 最后, 通过 一个实际例子说明了所提理论的实用性和可行性.     

Robustness of non-linear stochastic optimal control for quasi-Hamiltonian systems with parametric uncertainty

The robustness of non-linear stochastic optimal control for quasi-Hamiltonian systems with uncertain parameters is studied. Based on the independence of uncertain parameters and stochastic excitations, the non-linear stochastic optimal control for the nominal quasi-Hamiltonian system with average-value parameters is first obtained by using the stochastic averaging method and stochastic dynamical programming principle. Then, the means and standard deviations of root-mean-square responses, control effectiveness and control efficiency for the uncertain quasi-Hamiltonian system are calculated by using the stochastic averaging method and the probabilistic analysis. By introducing the sensitivity of the variation coefficients of controlled root-mean-square responses, control effectiveness and control efficiency to those of uncertain parameters, the robustness of the non-linear stochastic optimal control is evaluated. Two examples are given to illustrate the proposed control procedure and its robustness.     

A stochastically averaged optimal control strategy for quasi-Hamiltonian systems with actuator saturation

A stochastic optimal control strategy for quasi-Hamiltonian systems with actuator saturation is proposed based on the stochastic averaging method and stochastic dynamical programming principle. First, the partially completed averaged Itô stochastic differential equations for the energy processes of individual degree of freedom are derived by using the stochastic averaging method for quasi-Hamiltonian systems. Then, the dynamical programming equation is established by applying the stochastic dynamical programming principle to the partially completed averaged Itô equations with a performance index. The saturated optimal control consisting of unbounded optimal control and bounded bang-bang control is determined by solving the dynamical programming equation. Numerical results show that the proposed control strategy significantly improves the control efficiency and chattering attenuation of the corresponding bang-bang control.     

The stochastic maximum principle for optimal control problems of delay systems involving continuous and impulse controls

This paper is concerned with a Pontryagin’s maximum principle for stochastic optimal control problems of delay systems with random coefficients involving both continuous and impulse controls. This kind of control problems is motivated by some interesting phenomena arising from economics and finance. We establish a necessary maximum principle and a sufficient verification theorem by virtue of the duality and the convex analysis. To explain the theoretical results, we apply them to a production and consumption choice problem.     

Generalized maximum principle for optimal control of generalized state-space systems

This paper is mainly concerned with the derivation of the necessary conditions, called the ‘generalized maximum principle’, for the optimal control of generalized state-space systems with a more general form. By making use of a method of the modern calculus of variations which has been used for the proof of Pontryagin's maximum principle, the generalized maximum principle is derived, and some problems related to this principle are discussed in detail. In addition, an illustrative example is given in the light of this principle.     

Robust finite horizon minimax filtering for discrete-time stochastic uncertain systems

We study a finite-horizon robust minimax filtering problem for time-varying discrete-time stochastic uncertain systems. The uncertainty in the system is characterized by a set of probability measures under which the stochastic noises, driving the system, are defined. The optimal minimax filter has been found by applying techniques of risk-sensitive LQG control. The structure and properties of resulting filter are analyzed and compared to H_∞ and Kalman filters.     

一类不确定系统的最优极小极大鲁棒控制

讨论一类不确定系统的极小极大鲁棒动态输出反馈控制问题.给出不确定系统的极小极大鲁棒控制的定义.利用线性矩阵不等式(LMI)处理方法和Lyapunov稳定性理论,得到了在干扰和不确定性最大的情形下极小极大输出反馈控制器存在的充分条件.引入凸优化技术, 求得最优极小极大控制器.它不仅保证闭环系统渐近稳定, 且使得闭环系统性能指标的上界最小.仿真算例说明了所设计的控制器具有较强的干扰抑制功能.     

Robust optimal control for minimax stochastic linear quadratic problem

The robust maximum principle applied to the minimax linear quadratic problem is derived for stochastic differential equations containing a control-dependent diffusion term. The parametric families of the first and second order adjoint stochastic processes are obtained to construct the corresponding Hamiltonian formalism. The Hamiltonian function used for the construction of the robust optimal control is shown to be equal to the sum of the standard stochastic Hamiltonians corresponding to each value of the uncertain parameter from a given finite set. The cost function is considered on a finite horizon (contains the mathematical expectation of both an integral and a terminal term) and on an infinite one (a time-averaged losses function). These problems belong to the class of minimax stochastic optimization problems. It is shown that the construction of the minimax optimal controller can be reduced to an optimization problem on a finitedimensional simplex and consists in the analysis of the dependence of Riccati equation solution on the weight parameters to be found.     

Stochastic optimal control via Bellman''s principle

This paper presents a strategy for finding optimal controls of non-linear systems subject to random excitations. The method is capable to generate global control solutions when state and control constraints are present. The solution is global in the sense that controls for all initial conditions in a region of the state space are obtained. The approach is based on Bellman's principle of optimality, the cumulant neglect closure method and the short-time Gaussian approximation. Problems with state-dependent diffusion terms, non-closeable hierarchies of moment equations for the states and singular state boundary condition are considered in the examples. The uncontrolled and controlled system responses are evaluated by creating a Markov chain with a control dependent transition probability matrix via the generalized cell mapping method. In all numerical examples, excellent controlled performances were obtained.     

Synthesis of minimax optimal controllers for uncertain time-delay systems with structured uncertainty

This paper is concerned with the design of robust state feedback controllers for a class of uncertain time-delay systems. The uncertainty is assumed to satisfy a certain integral quadratic constraint. The controller proposed is a minimax optimal controller in the sense that it minimizes the maximum value of a corresponding linear quadratic cost function over all admissible uncertainties. The controller leads to an absolutely stable closed loop uncertain system and is constructed by solving a finite dimensional parameter-dependent algebraic Riccati equation.     

Global maximum principle for the forward–backward stochastic optimal control problem with poisson jumps

This paper is concerned with the forward–backward stochastic optimal control problem with Poisson jumps. A necessary condition of optimality in the form of a global maximum principle as well as a sufficient condition of optimality are presented under the assumption that the diffusion and jump coefficients do not contain the control variable, and the control domain need not be convex. The case where there are some state constraints is also discussed. A financial example is discussed to illustrate the application of our result. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

不确定高次随机非线性系统的自适应控制

针对一类含有噪声干扰和非线性参数的高次随机非线性系统,研究了依概率全局自适应稳定问题.在噪声的协方差未知的情况下,利用自适应增加幂积分方法和参数分离技术,提出了一种反馈占优设计方法并构造了一个光滑自适应控制器.该控制器能保证闭环系统依概率全局稳定,并且系统的状态几乎必然收敛到零.仿真例子验证了控制方案的有效性.     

Fully probabilistic control for uncertain nonlinear stochastic systems

This paper develops a novel probabilistic framework for stochastic nonlinear and uncertain control problems. The proposed framework exploits the Kullback–Leibler divergence to measure the divergence between the distribution of the closed-loop behavior of a dynamical system and a predefined ideal distribution. To facilitate the derivation of the analytic solution of the randomized controllers for nonlinear systems, transformation methods are applied such that the dynamics of the controlled system becomes affine in the state and control input. Additionally, knowledge of uncertainty is taken into consideration in the derivation of the randomized controller. The derived analytic solution of the randomized controller is shown to be obtained from a generalized state-dependent Riccati solution that takes into consideration the state- and control-dependent functional uncertainty of the controlled system. The proposed framework is demonstrated on an inverted pendulum on a cart problem, and the results are obtained.     

Towards an approach to stochastic adaptive control using the maximum entropy principle

The crucial problem in non-linear stochastic adaptive systems via the certainly equivalence principle is the estimation of the disturbances, which is essentially a non-linear estimation. The present paper focuses mainly on this aspect of adaptation, and the basic idea is of using the maximum entropy principle together with constraints suitably chosen. In this way one proposes a new technique for solving the Fokker-Plank-Kolmogorov equation and two new techniques for determining the conditional probability density of a random disturbance in a stochastic process. Then an adiabatic elimination is proposed, which applies when the system is slowly varying with respect to the external parameter. Finally, one shows how the dynamic equations of the state moments, combined with a linearization technique, can be utilized to analyse a broad class of non-linear stochastic systems involving random disturbances with small variances.