Stochastic maximum principle for optimal control problems involving delayed systems

Dear editor, The main objective of this study is to investigate one type of stochastic optimal control problem for a delayed system using the maximum principle ...     

带Markov跳的离散时间随机控制系统的最大值原理

本文研究一类同时含有Markov跳过程和乘性噪声的离散时间非线性随机系统的最优控制问题, 给出并证明了相应的最大值原理. 首先, 利用条件期望的平滑性, 通过引入具有适应解的倒向随机差分方程, 给出了带有线性差分方程约束的线性泛函的表示形式, 并利用Riesz定理证明其唯一性. 其次, 对带Markov跳的非线性随机控制系统, 利用针状变分法, 对状态方程进行一阶变分, 获得其变分所满足的线性差分方程. 然后, 在引入Hamilton函数的基础上, 通过一对由倒向随机差分方程刻画的伴随方程, 给出并证明了带有Markov跳的离散时间非线性随机最优控制问题的最大值原理, 并给出该最优控制问题的一个充分条件和相应的Hamilton-Jacobi-Bellman方程. 最后, 通过 一个实际例子说明了所提理论的实用性和可行性.     

Stochastic minimax optimal control strategy for uncertain quasi-Hamiltonian systems using stochastic maximum principle

A stochastic minimax optimal control strategy for uncertain quasi-Hamiltonian systems is proposed based on the stochastic averaging method, stochastic maximum principle and stochastic differential game theory. First, the partially completed averaged Itô stochastic differential equations are derived from a given system by using the stochastic averaging method for quasi-Hamiltonian systems with uncertain parameters. Then, the stochastic Hamiltonian system for minimax optimal control with a given performance index is established based on the stochastic maximum principle. The worst disturbances are determined by minimizing the Hamiltonian function, and the worst-case optimal controls are obtained by maximizing the minimal Hamiltonian function. The differential equation for adjoint process as a function of system energy is derived from the adjoint equation by using the Itô differential rule. Finally, two examples of controlled uncertain quasi-Hamiltonian systems are worked out to illustrate the application and effectiveness of the proposed control strategy.     

The maximum principle for partially observed optimal control problems of mean-field FBSDEs

This paper is concerned with a partially observed optimal control problem described by mean-field forward and backward stochastic differential equations. Moreover, the control variable enters the diffusion coefficient and the control domain is non-convex. Utilising Girsanov's theorem as well as extended Ekeland's variational principle, a maximum principle is established in the form of Pontryagin's type. As an application, a linear-quadratic control problem is studied in terms of the stochastic filtering.     

Generalized maximum principle for optimal control of generalized state-space systems

This paper is mainly concerned with the derivation of the necessary conditions, called the ‘generalized maximum principle’, for the optimal control of generalized state-space systems with a more general form. By making use of a method of the modern calculus of variations which has been used for the proof of Pontryagin's maximum principle, the generalized maximum principle is derived, and some problems related to this principle are discussed in detail. In addition, an illustrative example is given in the light of this principle.     

Maximum principle for the stochastic optimal control problem with delay and application

In this paper, we consider an optimal control problem for the stochastic system described by stochastic differential equations with delay. We obtain the maximum principle for the optimal control of this problem by virtue of the duality method and the anticipated backward stochastic differential equations. Our results can be applied to a production and consumption choice problem. The explicit optimal consumption rate is obtained.     

Stochastic maximum principle for delayed doubly stochastic control systems and their applications

ABSTRACT

In this paper, we investigate the optimal control problems for delayed doubly stochastic control systems. We first discuss the existence and uniqueness of the delayed doubly stochastic differential equation by martingale representation theorem and contraction mapping principle. As a necessary condition of the optimal control, we deduce a stochastic maximum principle under some assumption. At the same time, a sufficient condition of optimality is obtained by using the duality method. At the end of the paper, we apply our stochastic maximum principle to a class of linear quadratic optimal control problem and obtain the explicit expression of the optimal control.     

Approximate discrete maximum principle for the difference approximation of optimal periodic control problems

Distinctive features of the difference approximation for optimal periodic control (OPC) problems are considered. It is shown that the convergence conditions on approximate solutions imply consistency conditions on the time grid step and the accuracy of the periodicity constraint approximation such that the approximate discrete maximum principle holds for a sequence of discretized problems. A specific form of this principle when applied to the difference approximation method for OPC problems is analysed and computational methods are discussed. Application of this approach to a class of systems non-linearly dependent on control variables, and occurring in the chemical industry, is suggested.     

无限时间长时延网络控制系统的随机最优控制

考虑二次性能指标下线性网络控制系统的随机最优控制问题,建立了控制器为事件驱动时长时延线性网络控制系统的数学模型,证明了在无限时间情况下离散随机黎卡提代数方程解的存在性,设计出无限时间情况下线性网络控制系统的随机最优控制器,得到相应的最优性能指标的表达形式,并证明了相应的随机最优控制器可使网络控制系统均方指数稳定.最后以网络控制下的倒立摆为对象进行仿真研究,仿真结果表明该方法的正确性和有效性.     

A maximum principle for free endtime optimal control problems withdata discontinuous in time

Recently, a maximum principle has been proved for free-endtime problems when the dynamics and the control constraint set are assumed to be merely measurable with respect to the time variable. A new simple derivation of this optimality condition is given from standard fixed-endtime results. It involves a generalization of the customary boundary condition on the maximized Hamiltonian function, evaluated along the optimal state trajectory and costate function. The hypotheses under which the maximum principle is proved permit unilateral state constraints, data merely Lipschitz continuous in the state variable, and endpoint constraints expressed as general set inclusions. The primary concern is with methodology and setting down a simple proof of the maximum principle for problems with data measurable in the time variable     

The maximum principle for stochastic differential systems with general cost functional

In this paper, under the framework of Fréchet derivatives, we study a stochastic optimal control problem driven by a stochastic differential equation with general cost functional. By constructing a series of first-order and second-order adjoint equations, we establish the stochastic maximum principle and get the related Hamilton systems.     

Gradient refinement methods for optimal impulse control problems

An optimal control problem is considered for a system described by a differential equation with measures; a certain constraint is imposed on the total variation of controlmeasure. Involving the method of discontinuous time reparameterization, an interpretation is performed for the procedures of weak control variation in an auxiliary reduced problem, and new refinement methods are developed for impulsive processes. An example is provided.     

Global maximum principle for the forward–backward stochastic optimal control problem with poisson jumps

This paper is concerned with the forward–backward stochastic optimal control problem with Poisson jumps. A necessary condition of optimality in the form of a global maximum principle as well as a sufficient condition of optimality are presented under the assumption that the diffusion and jump coefficients do not contain the control variable, and the control domain need not be convex. The case where there are some state constraints is also discussed. A financial example is discussed to illustrate the application of our result. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Minimum and maximum delay problems in real-time systems

We consider a finite-state system with a finite number of clocks, where the transitions may depend on the values of the clocks, and may reset some of the clocks. We address the complexity and provide algorithms for the following problems. Suppose that the system starts from a given current state with a given assignment of values to the clocks. Can a given target state ever appear in the history of the system? What is the earliest time it can appear? What is the latest time it can appear?     

Pontryagin’s maximum principle in optimal control problems of orientation of a spacecraft

Pontryagin’s maximum principle is used for solution of topical problems of spacecraft motion control. The dynamic optimal control problem of space orientation of a spacecraft from an arbitrary initial to a given final angular position with minimization of the turning time is studied in detail. The solution to the formulated problem is obtained and numerical expressions for synthesis of optimal control program are given. Results of mathematical simulation of the dynamics of motion of a spacecraft at optimal control are presented; these results demonstrate practical feasibility of the developed control algorithm.     

Penalty method and maximum principle for nonsmooth variable-structure control problems

We consider the penalty method for a nonsmooth minimax control problem with interdependent variables. A scheme for the derivation of necessary optimality conditions and their application for the analysis of a particular technological process is described.Translated from Kibernetika i Sistemnyi Analiz, No. 3, pp. 125–130, May–June 1992.     

Joint maximum principle in the problem of synthesizing an optimal control of nonlinear systems

The problem of optimization of dynamic systems is considered. It is shown that, unlike the well-known methods of optimal control, through use of the maximum principle it becomes possible to synthesize an efficient control law that substantially reduces computational complexity, as is demonstrated by the results of numerical simulation.