Infinite time horizon nonzero-sum linear quadratic stochastic differential games with state and control-dependent noise

This paper discusses the infinite time horizon nonzero-sum linear quadratic （LQ） differential games of stochastic systems governed by Itoe＇s equation with state and control-dependent noise. First, the nonzero-sum LQ differential games are formulated by applying the results of stochastic LQ problems. Second, under the assumption of mean-square stabilizability of stochastic systems, necessary and sufficient conditions for the existence of the Nash strategy are presented by means of four coupled stochastic algebraic Riccati equations. Moreover, in order to demonstrate the usefulness of the obtained results, the stochastic H-two/H-infinity control with state, control and external disturbance-dependent noise is discussed as an immediate application.     

A new solution property in optimal control: The lens

We consider a new property of an optimal control problem called a lens.A lens is an interior point in the state-control phase plane where—given the value of the state variable—there is only one control value satisfying the necessary optimality conditions and—given the value of the control variable—there is only one state value satisfying the necessary optimality conditions.We build a simple model that generates a lens and give necessary and sufficient conditions under which a lens occurs.     

Decentralized optimality conditions of stochastic differential decision problems via Girsanov’s measure transformation

In this paper, we apply two methods to derive necessary and sufficient decentralized optimality conditions for stochastic differential decision problems with multiple Decision Makers (DMs), which aim at optimizing a common pay-off, based on the notions of decentralized global optimality and decentralized person-by-person (PbP) optimality. Method 1: We utilize the stochastic maximum principle to derive necessary and sufficient conditions which consist of forward and backward Stochastic Differential Equations (SDEs), and conditional variational Hamiltonians, conditioned on the information structures of the DMs. The sufficient conditions for decentralized PbP optimality are local conditions, closely related to the necessary conditions for decentralized PbP optimality. However, under certain convexity condition on the Hamiltonian, and a global version of the sufficient conditions for decentralized PbP optimality, we show decentralized global optimality. Method 2: We utilize the value processes of decentralized PbP optimal policies, we relate them to solutions of backward SDEs, we identify sufficient conditions for decentralized PbP optimality, and we show these are precisely those derived via the maximum principle. For both methods, as usual, we utilize Girsanov’s theorem to transform the initial decentralized stochastic optimal decision problems, to equivalent decentralized stochastic optimal decision problems on a reference probability space, in which the controlled process and the information processes which generate part of the information structures of the DMs, are independent of any of the decisions.     

Stability of θ-schemes for partial differential equations with piecewise constant arguments of alternately retarded and advanced type

ABSTRACT

This paper deals with the analytical and numerical stability of a partial differential equation with piecewise constant arguments of alternately retarded and advanced type. Firstly, the theory of separation of variables in matrix form and the Fourier method are implemented to achieve the sufficient conditions under which the analytic solution is asymptotically stable. Secondly, the discrete equation is obtained by applying the θ-schemes to the original continuous equation, the sufficient conditions for the asymptotic stability of numerical solution are also shown when the mesh ratio satisfying certain conditions. Finally, some numerical experiments for verifying the theoretical results are provided.     

关于定量与定性微分对策

本文将文献[2,3,4,5,7]中的方法加以发展,用来解决一类定量和定性微分对策问题. 对于定量对策,我们推出最优策略(u,v)所应满足的必要条件,即"双方极值原理".对于定 性对策,也得到最优策略(u,v)的必要条件、且不必如文献[1]中那样限于"小范围".并确定了 组成界栅(barrier)的轨线的方程. 还讨论了一些其他问题,如充分条件、目标集的更一般的形式、定性对策与能控性问题间 的关系等. 可见,这种方法是一种可用来解决多种类型的最优控制和微分对策问题的有力工具. 文中附有二例.     

Three-level incentive schemes using follower's strategies in differential games

We deal with three-level incentive differential games in which first and second leaders have access not only to slate information but also to information on follower's strategies. We derive sufficient conditions for three-level incentive schemes using information on follower's strategies in both linear and non-linear differential games, and show that three-level incentive schemes using information on follower's strategies depend on an initial state value.     

The relaxed optimal control problem for Mean-Field SDEs systems and application

The present study deals with a new approach of optimal control problems where the state equation is a Mean-Field stochastic differential equation, and the set of strict (classical) controls need not be convex and the diffusion coefficient depends on the term control. Our consideration is based on only one adjoint process, and the necessary conditions as well as a sufficient condition for optimality in the form of a relaxed maximum principle are obtained, with application to Linear quadratic stochastic control problem with mean-field type.     

A note on controllability,observability and sufficient coordinates in linear differential games†

This note examines the notions of controllability, observability and sufficient coordinates in linear differential games as applicable to a ‘ playable ’ pair of strategics.     

Optimality conditions and a solution scheme for fractional optimal control problems

We formulate necessary conditions for optimality in Optimal control problems with dynamics described by differential equations of fractional order (derivatives of arbitrary real order). Then by using an expansion formula for fractional derivative, optimality conditions and a new solution scheme is proposed. We assumed that the highest derivative in the differential equation of the process is of integer order. Two examples are treated in detail.     

Weakly monotone solutions of the Hamilton-Jacobi inequality and optimality conditions with positional controls

We obtain necessary global optimality conditions for classical optimal control problems based on positional controls. These controls are constructed with classical dynamical programming but with respect to upper (weakly monotone) solutions of the Hamilton-Jacobi equation instead of a Bellman function. We put special emphasis on the positional minimum condition in Pontryagin formalism that significantly strengthens the Maximum Principle for a wide class of problems and can be naturally combined with first order sufficient optimality conditions with linear Krotov’s function. We compare the positional minimum condition with the modified nonsmooth Ka?kosz-Lojasiewicz Maximum Principle. All results are illustrated with specific examples.     

Stochastic maximum principle for delayed doubly stochastic control systems and their applications

ABSTRACT

In this paper, we investigate the optimal control problems for delayed doubly stochastic control systems. We first discuss the existence and uniqueness of the delayed doubly stochastic differential equation by martingale representation theorem and contraction mapping principle. As a necessary condition of the optimal control, we deduce a stochastic maximum principle under some assumption. At the same time, a sufficient condition of optimality is obtained by using the duality method. At the end of the paper, we apply our stochastic maximum principle to a class of linear quadratic optimal control problem and obtain the explicit expression of the optimal control.     

Differential games with open-loop saddle point conditions

Zero-sum differential games are considered in which one or both of the players are restricted to use open-loop control. It is shown that the first-order necessary conditions for such problems are identical to the first-order necessary conditions for the usual form of a differential game, where both players use closed-loop control laws. An investigation of the conjugate point condition for a special class of games shows that this condition is not the same but depends on the type of solution sought. For games where one or both of the players use open-loop control, there are two conjugate point conditions that must be satisfied. This differs from games in which both players use closed-loop control, where there is only one conjugate point necessary condition.     

A risk‐sensitive stochastic maximum principle for fully coupled forward‐backward stochastic differential equations with applications

In this paper, we are interested in the problem of optimal control where the system is given by a fully coupled forward‐backward stochastic differential equation with a risk‐sensitive performance functional. As a preliminary step, we use the risk neutral which is an extension of the initial control system where the admissible controls are convex, and an optimal solution exists.Then, we study the necessary as well as sufficient optimality conditions for risk sensitive performance. At the end of this work, we illustrate our main result by giving an example that deals with an optimal portfolio choice problem in financial market, specifically the model of control cash flow of a firm or project where, for instance, we can set the model of pricing and managing an insurance contract.     

Sufficient conditions of optimality for stochastic systems withcontrollable diffusions

This paper studies optimal controls for systems governed by Ito's stochastic differential equations. Both the drift and diffusion terms of the equations are allowed to depend on controls, and the systems are allowed to be degenerate. It is shown that the necessary conditions of optimality, namely, the maximum conditions in terms of the “ℋ-function” (which is a generalization of the usual Hamiltonian and is quadratic with respect to the diffusion coefficients), along with some convexity conditions, constitute sufficient conditions of optimality for such controlled systems     

Quadratic games and H∞-type problems for time varying systems

H^∞-type control problems with state feedback are considered for time varying systems. First, general quadratic differential games are studied and then the relationship between H^∞-problems and differential games is clarified. A necessary and sufficient condition for the existence of a stabilizing controller, such that the norm of the operator from the disturbance to the output is less than a prescribed number, is given in terms of a Riccati equation. Earlier results on time invariant systems are found to be special cases of our main results. The finite horizon problem is also studied.     

Non-smooth and non-convex problems in discrete optimal control

A class of discrete control problems described only by locally Lipschitz functions is studied from the point of view of necessary optimality conditions. Moreover, it is assumed that state and control constraints are given implicitly as general sets being approximated by the respective ‘ generalized ’ tangent or normal cone. To investigate these types of non-differentiable optimization problems some basic facts of the so called non-smooth analysis have to be applied. The crucial role is played by a ‘ generalized ’ gradient of a locally Lipschitz function. Using these concepts one is able to formulate necessary optimality conditions in a fairly general setting. However, to obtain necessary conditions in a more familiar form an alternative definition of a partial generalized gradient ia explored. Some special cases of discrete control problems are studied separately. In addition, also the maximum principle formulation of necessary conditions is investigated and a question of possible extensions is briefly discussed     

An approach to a singular optimal control problem for non-linear systems using differential form theory

By applying differential form theory, we consider the singular control problem for non-linear systems with control variables appearing linearly in both the system dynamics and the performance index. First, we derive necessary conditions of singular optimality for a single-input system, including the relation to the Euler-Poisson equation and to the generalized Legendre-Clebsch condition. Defining the degree of singularity, we develop necessary conditions satisfied by the singular trajectory embedded in a reduced space. For a time-invariant system, we clarify the relation between the dynamic and the related static optimality. Second, we derive necessary conditions for singular optimality for a multi-input system where the dimension of the control vector is equal to that of the state space. We show that the Shima-Sawaragi condition for the optimality of boundary controls and the generalized Legendre-Clebsch condition are obtained from these conditions. The results are also applied to the analysis of a time-invariant system.     

Games of simple pursuit and approach on the manifolds

Two differential games in feedback strategies [1], [2] are considered, in which players are velocity-controlled points of a Riemannian manifold. The game of pursuit is formulated for the case when the pursuer has advantage in speed. Otherwise the game of approach is considered, i.e. the cost-function is the minimal distance between players during the infinite time-interval of motion. Since the restrictions for the velocities are homogenous, geodesic lines have an important role for optimal paths' construction. The main difference between manifold and Euclidean space is non-uniqueness of the minimal geodesic, connecting two points of the manifold. The analysis of this paper is restricted to the manifolds, which have no more than two minimal geodesics. This gives rise to the singularities of dispersal, equivocal and universal types. Local necessary optimality conditions are found. The players' optimal behaviour in general position is shown to be a (regular) motion along geodesics. The domain, where the latter lie on the geodesic curve, connecting the players, is called the primary domain. A sufficient condition is found for the whole game space to be the primary domain, as is the case in Euclidean space. Necessary conditions are formulated for the existence of singular paths, which are envelopes of the geodesics. The equations of singular motion are obtained and shown to be a generalized Hamiltonian type. An algorithm is suggested for the construction of the optimal paths in the vicinity of a singular surface, its efficiency is demonstrated by complete solutions of both games on a two-dimensional cone. For other approaches to similar game problems see [1]–[5].     

Dynamic Games for Stochastic Systems with Delay

This paper discusses dynamic games for a class of linear stochastic delay systems governed by Itô's stochastic differential equation. The Pareto and Nash strategies are developed by solving cross‐coupled matrix inequalities. To obtain these strategy sets, new cross‐coupled algebraic equations (CSAEs) are established on the basis of the Karush‐Kuhn‐Tucker (KKT) conditions, which constitute the necessary conditions. It is noteworthy that the state feedback strategies can be obtained by solving the linear matrix inequality (LMI) recursively. Finally, a numerical example showing the effectiveness of the proposed methods and the attained cost bounds is described.