带Markov跳的离散时间随机控制系统的最大值原理

本文研究一类同时含有Markov跳过程和乘性噪声的离散时间非线性随机系统的最优控制问题, 给出并证明了相应的最大值原理. 首先, 利用条件期望的平滑性, 通过引入具有适应解的倒向随机差分方程, 给出了带有线性差分方程约束的线性泛函的表示形式, 并利用Riesz定理证明其唯一性. 其次, 对带Markov跳的非线性随机控制系统, 利用针状变分法, 对状态方程进行一阶变分, 获得其变分所满足的线性差分方程. 然后, 在引入Hamilton函数的基础上, 通过一对由倒向随机差分方程刻画的伴随方程, 给出并证明了带有Markov跳的离散时间非线性随机最优控制问题的最大值原理, 并给出该最优控制问题的一个充分条件和相应的Hamilton-Jacobi-Bellman方程. 最后, 通过 一个实际例子说明了所提理论的实用性和可行性.     

Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems

This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.     

Robust stabilisation of discrete-time time-varying linear systems with Markovian switching and nonlinear parametric uncertainties

In this paper, the problem of the robustness of the stability of a discrete-time linear stochastic system is addressed. The nominal plant is described by a discrete-time time-varying linear system subject to random jumping according with a non-homogeneous Markov chain with a finite number of states. The class of admissible uncertainties consists of multiplicative white noise type perturbations with unknown intensity. It is assumed that the intensity of white noise type perturbations is modelled by unknown nonlinear functions subject to linear growth conditions. The class of admissible controls consists of stabilising state feedback control laws. We show that the best robustness performance is achieved by the stability provided by a state feedback design based on the stabilising solution of a suitable discrete-time Riccati-type equation.     

Iterative algorithm to compute the maximal and stabilising solutions of a general class of discrete-time Riccati-type equations

In this article an iterative method to compute the maximal solution and the stabilising solution, respectively, of a wide class of discrete-time nonlinear equations on the linear space of symmetric matrices is proposed. The class of discrete-time nonlinear equations under consideration contains, as special cases, different types of discrete-time Riccati equations involved in various control problems for discrete-time stochastic systems. This article may be viewed as an addendum of the work of Dragan and Morozan (Dragan, V. and Morozan, T. (2009), ‘A Class of Discrete Time Generalized Riccati Equations’, Journal of Difference Equations and Applications, first published on 11 December 2009 (iFirst), doi: 10.1080/10236190802389381) where necessary and sufficient conditions for the existence of the maximal solution and stabilising solution of this kind of discrete-time nonlinear equations are given. The aim of this article is to provide a procedure for numerical computation of the maximal solution and the stabilising solution, respectively, simpler than the method based on the Newton–Kantorovich algorithm.     

A risk‐sensitive stochastic maximum principle for fully coupled forward‐backward stochastic differential equations with applications

In this paper, we are interested in the problem of optimal control where the system is given by a fully coupled forward‐backward stochastic differential equation with a risk‐sensitive performance functional. As a preliminary step, we use the risk neutral which is an extension of the initial control system where the admissible controls are convex, and an optimal solution exists.Then, we study the necessary as well as sufficient optimality conditions for risk sensitive performance. At the end of this work, we illustrate our main result by giving an example that deals with an optimal portfolio choice problem in financial market, specifically the model of control cash flow of a firm or project where, for instance, we can set the model of pricing and managing an insurance contract.     

Necessary and sufficient conditions of risk‐sensitive optimal control and differential games for stochastic differential delayed equations

In this paper, we consider risk‐sensitive optimal control and differential games for stochastic differential delayed equations driven by Brownian motion. The problems are related to robust stochastic optimization with delay due to the inherent feature of the risk‐sensitive objective functional. For both problems, by using the logarithmic transformation of the associated risk‐neutral problem, the necessary and sufficient conditions for the risk‐sensitive maximum principle are obtained. We show that these conditions are characterized in terms of the variational inequality and the coupled anticipated backward stochastic differential equations (ABSDEs). The coupled ABSDEs consist of the first‐order adjoint equation and an additional scalar ABSDE, where the latter is induced due to the nonsmooth nonlinear transformation of the adjoint process of the associated risk‐neutral problem. For applications, we consider the risk‐sensitive linear‐quadratic control and game problems with delay, and the optimal consumption and production game, for which we obtain explicit optimal solutions.     

Optimal control of mean-field backward doubly stochastic systems driven by Itô-Lévy processes

ABSTRACT

In this paper, we introduce a new class of backward doubly stochastic differential equations (in short BDSDE) called mean-field backward doubly stochastic differential equations (in short MFBDSDE) driven by Itô-Lévy processes and study the partial information optimal control problems for backward doubly stochastic systems driven by Itô-Lévy processes of mean-field type, in which the coefficients depend on not only the solution processes but also their expected values. First, using the method of contraction mapping, we prove the existence and uniqueness of the solutions to this kind of MFBDSDE. Then, by the method of convex variation and duality technique, we establish a sufficient and necessary stochastic maximum principle for the stochastic system. Finally, we illustrate our theoretical results by an application to a stochastic linear quadratic optimal control problem of a mean-field backward doubly stochastic system driven by Itô-Lévy processes.     

A stochastic maximum principle for systems with jumps, with applications to finance

We consider a stochastic control problem with linear dynamics with jumps, convex cost criterion, and convex state constraint, in which the control enters the drift, the diffusion, and the jump coefficients. We allow these coefficients to be random, and do not impose any L^p-bounds on the control.

We obtain a stochastic maximum principle for this model that provides both necessary and sufficient conditions of optimality. This is the first version of the stochastic maximum principle that covers the consumption–investment problem in which there are jumps in the price system.     

Decentralized linear–quadratic–Gaussian control of networked control systems with asymmetric information

The decentralized linear–quadratic–Gaussian (LQG) control problem for networked control systems (NCSs) with asymmetric information is investigated, where controller 1 shares its historical information with controller 2, and not vice versa. The asymmetry of the information structure leads to the coupling between controller 2 and estimator 1, and hence the classical separation principle fails. Through the assumption of linear control strategy, the coupling between controller 2 and estimator 1 (CCE) is decoupled, but the estimation gain is still coupled with the control gain. It is noted that the control gain conforms to the backward Riccati equation while estimation gain abides by the forward equation, which is computationally challenging. Applying the stochastic maximum principle, the solvability of the decentralized LQG control problem is reduced to that of corresponding forward and backward stochastic difference equations (FBSDEs). Further, necessary and sufficient conditions for the solvability of optimal control problem are presented by two Riccati equations, one of which is nonsymmetric. Moreover, a novel iterative forward method is proposed to calculate the coupled backward control gain and forward estimation gain.     

On optimal control of mean-field stochastic systems driven by Teugels martingales via derivative with respect to measures

ABSTRACT

This paper deals with partial information stochastic optimal control problem for general controlled mean-field systems driven by Teugels martingales associated with some Lévy process having moments of all orders, and an independent Brownian motion. The coefficients of the system depend on the state of the solution process as well as of its probability law and the control variable. We establish a set of necessary conditions in the form of Pontryagin maximum principle for the optimal control. We also give additional conditions, under which the necessary optimality conditions turn out to be sufficient. The proof of our result is based on the derivative with respect to the probability law by applying Lions derivatives and a corresponding Itô formula. As an application, conditional mean-variance portfolio selection problem in incomplete market, where the system is governed by some Gamma process is studied to illustrate our theoretical results.