A Computational Method for Stochastic Optimal Control Problems in Financial Mathematics

Principle of optimality or dynamic programming leads to derivation of a partial differential equation (PDE) for solving optimal control problems, namely the Hamilton‐Jacobi‐Bellman (HJB) equation. In general, this equation cannot be solved analytically; thus many computing strategies have been developed for optimal control problems. Many problems in financial mathematics involve the solution of stochastic optimal control (SOC) problems. In this work, the variational iteration method (VIM) is applied for solving SOC problems. In fact, solutions for the value function and the corresponding optimal strategies are obtained numerically. We solve a stochastic linear regulator problem to investigate the applicability and simplicity of the presented method and prove its convergence. In particular, for Merton's portfolio selection model as a problem of portfolio optimization, the proposed numerical method is applied for the first time and its usefulness is demonstrated. For the nonlinear case, we investigate its convergence using Banach's fixed point theorem. The numerical results confirm the simplicity and efficiency of our method.     

Determination of optimal feedback terminal controllers for general boundary conditions using generating functions

Given a nonlinear system and a performance index to be minimized, we present a general approach to expressing the finite time optimal feedback control law applicable to different types of boundary conditions. Starting from the necessary conditions for optimality represented by a Hamiltonian system, we solve the Hamilton-Jacobi equation for a generating function for a specific canonical transformation. This enables us to obtain the optimal feedback control for fundamentally different sets of boundary conditions only using a series of algebraic manipulations and partial differentiations. Furthermore, the proposed approach reveals an insight that the optimal cost functions for a given dynamical system can be decomposed into a single generating function that is only a function of the dynamics plus a term representing the boundary conditions. This result is formalized as a theorem. The whole procedure provides an advantage over methods rooted in dynamic programming, which require one to solve the Hamilton-Jacobi-Bellman equation repetitively for each type of boundary condition. The cost of this favorable versatility is doubling the dimension of the partial differential equation to be solved.     

带有运行成本函数的Hamilton-Jacobi可达性分析

水平集方法将可达集表示为Hamilton-Jacobi方程解的零水平集,保存多个不同时间范围的可达集则需要保存Hamilton-Jacobi方程在多个时刻的解,这不仅需要消耗大量的存储空间还为控制律的设计造成了困难.针对这些局限性,提出了一种改进的基于Hamilton-Jacobi方程的可达集表示方法.该方法在Hamilton-Jacobi方程中加入了一项运行成本函数,可以用同一个时刻的解的多个非零水平集表示多个不同时间范围的可达集,极大地节省了存储空间并为控制律的设计提供了便利.为了求解所构造的带有运行成本函数的Hamilton-Jacobi方程,采用了一种基于递归和插值的方法.最后,通过一些数值算例验证了所提出的方法的精确性、在存储空间方面的优越性以及设计的控制律的有效性.     

Solving a category of two-dimensional fractional optimal control problems using discrete Legendre polynomials

In this paper, we formulate a numerical method to approximate the solution of two-dimensional optimal control problem with a fractional parabolic partial differential equation (PDE) constraint in the Caputo type. First, the optimal conditions of the optimal control problems are derived. Then, we discretize the spatial derivatives and time derivatives terms in the optimal conditions by using shifted discrete Legendre polynomials and collocations method. The main idea is simplifying the optimal conditions to a system of algebraic equations. In fact, the main privilege of this new type of discretization is that the numerical solution is directly and globally obtained by solving one efficient algebraic system rather than step-by-step process which avoids accumulation and propagation of error. Several examples are tested and numerical results show a good agreement between exact and approximate solutions.     

SDP-based approximation of stabilising solutions for periodic matrix Riccati differential equations

Numerically finding stabilising feedback control laws for linear systems of periodic differential equations is a nontrivial task with no known reliable solutions. The most successful method requires solving matrix differential Riccati equations with periodic coefficients. All previously proposed techniques for solving such equations involve numerical integration of unstable differential equations and consequently fail whenever the period is too large or the coefficients vary too much. Here, a new method for numerical computation of stabilising solutions for matrix differential Riccati equations with periodic coefficients is proposed. Our approach does not involve numerical solution of any differential equations. The approximation for a stabilising solution is found in the form of a trigonometric polynomial, matrix coefficients of which are found solving a specially constructed finite-dimensional semidefinite programming (SDP) problem. This problem is obtained using maximality property of the stabilising solution of the Riccati equation for the associated Riccati inequality and sampling technique. Our previously published numerical comparisons with other methods shows that for a class of problems only this technique provides a working solution. Asymptotic convergence of the computed approximations to the stabilising solution is proved below under the assumption that certain combinations of the key parameters are sufficiently large. Although the rate of convergence is not analysed, it appeared to be exponential in our numerical studies.     

Dynamic programming and viscosity solutions for the optimal control of quantum spin systems

The purpose of this paper is to describe the application of the notion of viscosity solutions to solve the Hamilton-Jacobi-Bellman (HJB) equation associated with an important class of optimal control problems for quantum spin systems. The HJB equation that arises in the control problems of interest is a first-order nonlinear partial differential equation defined on a Lie group. Hence we employ recent extensions of the theory of viscosity solutions to Riemannian manifolds in order to interpret possibly non-differentiable solutions to this equation. Results from differential topology on the triangulation of manifolds are then used develop a finite difference approximation method for numerically computing the solution to such problems. The convergence of these approximations is proven using viscosity solution methods. In order to illustrate the techniques developed, these methods are applied to an example problem.     

Numerical Solution of Optimal Control Problems with Constant Control Delays

We investigate a class of optimal control problems that exhibit constant exogenously given delays in the control in the equation of motion of the differential states. Therefore, we formulate an exemplary optimal control problem with one stock and one control variable and review some analytic properties of an optimal solution. However, analytical considerations are quite limited in case of delayed optimal control problems. In order to overcome these limits, we reformulate the problem and apply direct numerical methods to calculate approximate solutions that give a better understanding of this class of optimization problems. In particular, we present two possibilities to reformulate the delayed optimal control problem into an instantaneous optimal control problem and show how these can be solved numerically with a state-of-the-art direct method by applying Bock’s direct multiple shooting algorithm. We further demonstrate the strength of our approach by two economic examples.      

Singular optimal control for stochastic linear quadratic singular system using ant colony programming

In this article, singular optimal control for stochastic linear singular system with quadratic performance is obtained using ant colony programming (ACP). To obtain the optimal control, the solution of matrix Riccati differential equation is computed by solving differential algebraic equation using a novel and nontraditional ACP approach. The obtained solution in this method is equivalent or very close to the exact solution of the problem. Accuracy of the solution computed by the ACP approach to the problem is qualitatively better. The solution of this novel method is compared with the traditional Runge Kutta method. An illustrative numerical example is presented for the proposed method.     

The stable discrete numerical solution of strongly coupled mixed partial differential systems

This paper is concerned with the discrete numerical solution of coupled partial differential mixed problems with non-Dirichlet coupled boundary value conditions. By application of a discrete separation of variables method, the proposed numerical solution of the problem turns out to be the exact solution of certain coupled partial difference system, appearing from the discretization of the continuous partial differential system. Our approach avoids the iterative solution of algebraic systems which appears when one uses simple discretization methods. Existence, stability and the construction of solutions are considered.     

Quadratic optimization of motion coordination and control

Algorithms for continuous-time quadratic optimization of motion control are presented. Explicit solutions to the Hamilton-Jacobi equation for optimal control of rigid-body motion are found by solving an algebraic matrix equation. The system stability is investigated according to Lyapunov function theory and it is shown that global asymptotic stability holds. How optimal control and adaptive control may act in concert in the case of unknown or uncertain system parameters is shown. The solution results in natural design parameters in the form of square weighting matrices, as known from linear quadratic optimal control. The proposed optimal control is useful both for motion control, trajectory planning, and motion analysis     

Aircraft Trajectory Optimization for Collision Avoidance Using Stochastic Optimal Control

Optimizing aircraft collision avoidance and performing trajectory optimization are the key problems in an air transportation system. This paper is focused on solving these problems by using a stochastic optimal control approach. The major contribution of this paper is a proposed stochastic optimal control algorithm to dynamically adjust and optimize aircraft trajectory. In addition, this algorithm accounts for random wind dynamics and convective weather areas with changing size. Although the system is modeled by a stochastic differential equation, the optimal feedback control for this equation can be computed as a solution of a partial differential equation, namely, an elliptic Hamilton‐Jacobi‐Bellman equation. In this paper, we solve this equation numerically using a Markov Chain approximation approach, where a comparison of three different iterative methods and two different optimization search methods are presented. Simulations show that the proposed method provides better performance in reducing conflict probability in the system and that it is feasible for real applications.     

基于Hamilton-Jacobi方程的飞行器机动动作可达集分析

为了给驾驶员完成标准机动动作提供决策支持, 提出一种使用哈密尔顿-雅克比(Hamilton-Jacobi)方程求解机动动作可行状态空间的研究方法.使用关键点将机动动作划分为不同阶段, 将各关键点的标准状态约束作为目标集, 逆时间求解目标集对应的可达集得到各阶段的边界状态范围, 目标集和可达集均由零水平集表示.使用该方法得到斤斗动作三维度运动模型下各阶段的可达集及斤斗动作的可行状态空间, 为了使运动模型的控制量与驾驶员实际操纵更为接近, 构建了以迎角变化率为控制量的四维度运动模型, 在此基础上对斤斗动作各阶段的可达集进行了分析.     

Hamilton-Jacobi inequalities and the optimality conditions in the problems of control with common end constraints

The canonical theory of the necessary and sufficient conditions for global optimality based on the sets of nonsmooth solutions of the differential Hamilton-Jacobi inequalities of two classes of weakly and strongly monotone Lyapunov type functions was developed. These functions enable one to estimate from above and below the objective functional of the optimal control problem and determine the internal and external approximations of the reachability set of the controlled dynamic system.     

Power-shaping control: Writing the system dynamics into the Brayton-Moser form

Power-shaping control is an extension of energy-balancing passivity-based control that is based on a particular form of the dynamics, the Brayton-Moser form. One of the main difficulties in this control approach is to write the dynamics in the suitable form since this requires the solution of a partial differential equation (PDE) system with an additional sign constraint. Here a general methodology is described for solving this partial differential equation system. The set of all solutions to the PDE system is given as the solution of a linear equation system. Furthermore a necessary condition is given so that a solution of the linear system which meets the sign condition exists. This methodology is illustrated on the case of a chemical reactor, where the physical knowledge of the system is used to find a suitable solution.     

Remarks on the application of dynamic programming to the optimal path timing of robot manipulators

In this paper, we investigate the use of the dynamic programming approach in the solution of the optimal path timing problem in robotics. This problem is computationally feasible because the path constraint reduces the dimension of the state in the problem to two. The Hamilton–Jacobi–Bellman equation of dynamic programming, a nonlinear first order partial differential equation, is presented and is solved approximately using finite difference methods. Numerical solution of this results in the optimal policy which can then be used to define the optimal path timing by numerical integration. Issues relating to the convergence of the numerical schemes are discussed, and the results are applied to an experimental SCARA manipulator. © 1998 John Wiley & Sons, Ltd.     

含扩散项不可靠生产系统最优生产控制的数值求解

针对含扩散项不可靠随机生产系统最优生产控制的优化命题, 采用数值解方法来求解该优化命题最优控制所满足的模态耦合的非线性偏微分HJB方程. 首先构造Markov链来近似生产系统状态演化, 并基于局部一致性原理, 把求解连续时间随机控制问题转化为求解离散时间的Markov决策过程问题, 然后采用数值迭代和策略迭代算法来实现最优控制数值求解过程. 文末仿真结果验证了该方法的正确性和有效性.     

Exact Boundary Condition for Semi-discretized Schrödinger Equation and Heat Equation in a Rectangular Domain

An approach to solve finite time horizon suboptimal feedback control problems for partial differential equations is proposed by solving dynamic programming equations on adaptive sparse grids. A semi-discrete optimal control problem is introduced and the feedback control is derived from the corresponding value function. The value function can be characterized as the solution of an evolutionary Hamilton–Jacobi Bellman (HJB) equation which is defined over a state space whose dimension is equal to the dimension of the underlying semi-discrete system. Besides a low dimensional semi-discretization it is important to solve the HJB equation efficiently to address the curse of dimensionality. We propose to apply a semi-Lagrangian scheme using spatially adaptive sparse grids. Sparse grids allow the discretization of the value functions in (higher) space dimensions since the curse of dimensionality of full grid methods arises to a much smaller extent. For additional efficiency an adaptive grid refinement procedure is explored. The approach is illustrated for the wave equation and an extension to equations of Schrödinger type is indicated. We present several numerical examples studying the effect the parameters characterizing the sparse grid have on the accuracy of the value function and the optimal trajectory.     

Hybrid Solution Method for Dynamic Programming Equations for MDOF Stochastic Systems

An optimal control problem is considered for a multi-degree-of-freedom (MDOF) system, excited by a white-noise random force. The problem is to minimize the expected response energy by a given time instantT by applying a vector control force with given bounds on magnitudes of its components. This problem is governed by the Hamilton-Jacobi-Bellman, or HJB, partial differential equation. This equation has been studied previously [1] for the case of a single-degree-of-freedom system by developing a hybrid solution. Specifically, an exact analitycal solution has been obtained within a certain outer domain of the phase plane, which provides necessary boundary conditions for numerical solution within a bounded in velocity inner domain, thereby alleviating problem of numerical analysis for an unbounded domain. This hybrid approach is extended here to MDOF systems using common transformation to modal coordinates. The multidimensional HJB equation is solved explicitly for the corresponding outer domain, thereby reducing the problem to a set of numerical solutions within bounded inner domains. Thus, the problem of bounded optimal control is solved completely as long as the necessary modal control forces can be implemented in the actuators. If, however, the control forces can be applied to the original generalized coordinates only, the resulting optimal control law may become unfeasible. The reason is the nonlinearity in maximization operation for modal control forces, which may lead to violation of some constraints after inverse transformation to original coordinates. A semioptimal control law is illustrated for this case, based on projecting boundary points of the domain of the admissible transformed control forces onto boundaries of the domain of the original control forces. Case of a single control force is considered also, and similar solution to the HJB equation is derived.     

基于LS-SVM求解非线性常微分方程组的近似解

提出了一种基于最小二乘支持向量机(LS-SVM)的改进方法求解非线性常微分方程组初值问题的近似解.利用径向基核函数(RBF)可导的特点对LS-SVM模型进行改进,将含核函数导数形式的LS-SVM模型转化为优化问题进行求解.方法可在原始对偶集中获得近似解的最佳表示,所得近似解连续可微,且精度较高.给出数值算例,通过与真实解的对比验证了所提方法的准确性和有效性.