A Numerical Method for Solving Stochastic Optimal Control Problems with Linear Control

We introduce a numerical method to solve stochastic optimal control problems which are linear in the control. We facilitate the idea of solving two-point boundary value problems with spline functions in order to solve the resulting dynamic programming equation. We then show how to effectively reduce the dimension in the proposed algorithm, which improves computational time and memory constraints. An example, motivated as an invest problem with uncertain cost, is provided, and the effectiveness of our method demonstrated.     

求解非线性控制系统最优控制的罚函数方法

本文在文献[3,4]的基础上,研究了控制系统是非线性的情形下,带外(内)罚函数的非受限最优控制问题Ako(Ak)趋于受限最优控制问题A的充分条件和必要条件.     

Solving Fixed Final Time Fractional Optimal Control Problems Using the Parametric Optimization Method

In this paper, the parametric optimization method is used to find optimal control laws for fractional systems. The proposed approach is based on the use for the fractional variational iteration method to convert the original optimal control problem into a nonlinear optimization one. The control variable is parameterized by unknown parameters to be determined, then its expression is substituted into the system state‐space model. The resulting fractional ordinary differential equations are solved by the fractional variational iteration method, which provides an approximate analytical expression of the closed‐form solution of the state equations. This solution is a function of time and the unknown parameters of the control law. By substituting this solution into the performance index, the original fractional optimal control problem reduces to a nonlinear optimization problem where the unknown parameters, introduced in the parameterization procedure, are the optimization variables. To solve the nonlinear optimization problem and find the optimal values of the control parameters, the Alienor global optimization method is used to achieve the global optimal values of the control law parameters. The proposed approach is illustrated by two application examples taken from the literature.     

解决最优控制问题的准梯度方法

讨论非线性最优控制问题构造的标准梯度方法的两种改进方法,文中的方法是以罚函数的有效近似为基础,与标准梯度方法相比,该方法对于非线性二次问题具有非局部的特点,同时给出了相关收敛结果。     

Solving Stochastic Transportation Network Protection Problems Using the Progressive Hedging-based Method

This research focuses on pre-disaster transportation network protection against uncertain future disasters. Given limited resources, the goal of the central planner is to choose the best set of network components to protect while allowing the network users to follow their own best-perceived routes in any resultant network configuration. This problem is formulated as a two-stage stochastic programming problem with equilibrium constraints, where the objective is to minimize the total expected physical and social losses caused by potential disasters. Developing efficient solution methods for such a problem can be challenging. In this work, we will demonstrate the applicability of progressive hedging-based method for solving large scale stochastic network optimization problems with equilibrium constraints. In the proposed solution procedure, we solve each modified scenario sub-problem as a mathematical program with complementary constraints and then gradually aggregate scenario-dependent solutions to the final optimal solution.     

Solving Optimal Control Problems on the Class of Step Functions

Linear optimal control problems with multipoint non-separated conditions with a quadratic performance criterion of control are analyzed. An approach with a violation of intermediate conditions is proposed. The original problem is reduced to the classical quadratic programming problem with the dimension determined by the number of intermediate points. __________ Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 153–162, July–August 2005.     

求解最优控制问题的Chebyshev-Gauss伪谱法

提出了一种求解最优控制问题的Chebyshev-Gauss伪谱法, 配点选择为Chebyshev-Gauss点. 通过比较非线性规划问题的Kaursh-Kuhn-Tucker条件和伪谱离散化的最优性条件, 导出了协态和Lagrange乘子的估计公式. 在状态逼近中, 采用了重心Lagrange插值公式, 并提出了一种简单有效的计算状态伪谱微分矩阵的方法. 该法的独特优势是具有良好的数值稳定性和计算效率. 仿真结果表明, 该法能够高精度地求解带有约束的复杂最优控制问题.     

Optimal Control of Nonlinear Stochastic Systems under Constraints: An Approximate Determination Method

An approximate method is developed for determining the optimal control for nonlinear stochastic systems under mixed equality- and inequality-type constraints on the parameters of the system, control functions, and phase coordinate in the presence of random parameters and additive and multiplicative noises. The method is based on the reduction of the initial stochastic problem to a deterministic problem for the cumulants of a random process described by stochastic differential equations.     

采用带时滞的输入—输出模型的随机最优控制

由于在系统模型中未引入时滞,以及计算量大等原因,尽管随机最优控制有很多优点,但未能成功地应用于过程控制。本文从新的角度,提出一种离散随机最优控制算法,该法采用带有时滞的输入-输出对象模型,而且无需求解Riccati方程和进行谱因子分解。本算法要求很少的计算量,因而易在微型计算机上实现。文中还对系统闭环特性进行分析。     

A Computational Method for Stochastic Optimal Control Problems in Financial Mathematics

Principle of optimality or dynamic programming leads to derivation of a partial differential equation (PDE) for solving optimal control problems, namely the Hamilton‐Jacobi‐Bellman (HJB) equation. In general, this equation cannot be solved analytically; thus many computing strategies have been developed for optimal control problems. Many problems in financial mathematics involve the solution of stochastic optimal control (SOC) problems. In this work, the variational iteration method (VIM) is applied for solving SOC problems. In fact, solutions for the value function and the corresponding optimal strategies are obtained numerically. We solve a stochastic linear regulator problem to investigate the applicability and simplicity of the presented method and prove its convergence. In particular, for Merton's portfolio selection model as a problem of portfolio optimization, the proposed numerical method is applied for the first time and its usefulness is demonstrated. For the nonlinear case, we investigate its convergence using Banach's fixed point theorem. The numerical results confirm the simplicity and efficiency of our method.     

Aircraft Trajectory Optimization for Collision Avoidance Using Stochastic Optimal Control

Optimizing aircraft collision avoidance and performing trajectory optimization are the key problems in an air transportation system. This paper is focused on solving these problems by using a stochastic optimal control approach. The major contribution of this paper is a proposed stochastic optimal control algorithm to dynamically adjust and optimize aircraft trajectory. In addition, this algorithm accounts for random wind dynamics and convective weather areas with changing size. Although the system is modeled by a stochastic differential equation, the optimal feedback control for this equation can be computed as a solution of a partial differential equation, namely, an elliptic Hamilton‐Jacobi‐Bellman equation. In this paper, we solve this equation numerically using a Markov Chain approximation approach, where a comparison of three different iterative methods and two different optimization search methods are presented. Simulations show that the proposed method provides better performance in reducing conflict probability in the system and that it is feasible for real applications.     

Lipschitzian Regularity of the Minimizing Trajectories for Nonlinear Optimal Control Problems

We consider the Lagrange problem of optimal control with unrestricted controls and address the question: under what conditions can we assure optimal controls are bounded? This question is related to one of Lipschitzian regularity of optimal trajectories, and the answer to it is crucial in closing the gap between the conditions arising in existence theory and necessary optimality conditions. Rewriting the Lagrange problem in a parametric form, we obtain a relation between the applicability conditions of the Pontryagin maximum principle to the latter problem and the Lipschitzian regularity conditions for the original problem. Under the standard hypotheses of coercivity of the existence theory, the conditions imply that the optimal controls are essentially bounded, assuring the applicability of the classical necessary optimality conditions like the Pontryagin maximum principle. The result extends previous Lipschitzian regularity results to cover optimal control problems with general nonlinear dynamics.     

具有不确定Wiener噪声随机非线性系统的自适应逆最优控制

针对一类具有不确定Wiener噪声扰动和未知定常参数的随机非线性系统,采用随机微分方程描述系统,基于Backstepping算法,利用随机控制Lyapunov函数,研究了自适应逆最优控制问题的可解定理,系统地给出了全局依概率渐近稳定和自适应逆最优控制策略的设计方法.这种方法可同时获得控制律和自适应律,仿真结果表明该控制算法的有效性.     

Optimal Control and Stochastic Simulation of Large Nonlinear Models with Rational Expectations

This paper presents a computationally fesible procedure for the optimalcontrol and stochastic simulation of large nonlinear models with rationalexpectations under the assumption of certainty equivalence.     

带不等式路径约束最优控制问题的惩罚函数法

控制变量参数化（Control variable parameterization,CVP）方法是目前求解流程工业中最优操作问题的主流数值方法,但如果问题中包含路径约束,特别是不等式路径约束时,CVP方法则需要考虑专门的处理手段.为了克服该缺点,本文提出一种基于L1精确惩罚函数的方法,能够有效处理关于控制变量、状态变量、甚至控制变量/状态变量复杂耦合形式下的不等式路径约束.此外,为了能使用基于梯度的成熟优化算法,本文还引进了最新出现的光滑化技巧对非光滑的惩罚项进行磨光.最终得到了能高效处理不等式路径约束的改进型CVP架构,并给出相应数值算法.经典的带不等式路径约束最优控制问题上的测试结果及与国外文献报道的比较研究表明：本文所提出的改进型CVP 架构及相应算法在精度和效率上兼有良好表现.     

一种采用方块脉冲函数求解延时最优控制问题的算法

本文采用方块脉冲函数方法,将线性延时系统二次型最优控制问题转化为函数极值问题,得到了最优控制规律的分段恒定解答,导出了一种不同迭代求解的简便算法.所提算法对小延时和大延时系统均有效.     

求解含复杂约束非线性最优控制问题的改进Gauss伪谱法

针对含有复杂约束条件的非线性最优控制问题,提出了一种改进的Gauss伪谱法 (Improved Gauss pseudospectral method, IGPM). 这类问题难以得到解析解,特别是有些问题不存在解析的模型, 一些参数只能通过查表得到,使得传统方法难以求解. 在传统的Gauss伪谱法的基础上,将非线性的终端状态积分约束等价地转化为线性形式,提出了IGPM, 通过协态映射定理可以计算出协态变量,检验最优性,使得IGPM具有间接法一样的精度. 并且给出了初始时刻协态变量和端点时刻控制变量的计算方法. 为了提高解的精度,基于IGPM提出了迭代算法, 最后将该算法应用于求解高超声速飞行器上升段轨迹优化问题,结果表明最优轨迹基本满足路径约束条件和最优性条件.     

Solving the p-hub Median Problem Under Intentional Disruptions Using Simulated Annealing

Hubs are special facilities designed to act as switching, transshipment and sorting points in various distribution systems. Since hub facilities concentrate and consolidate flows, disruptions at hubs could have large effects on the performance of a hub network. In this paper, we have formulated the multiple allocation p-hub median problem under intentional disruptions as a bi-level game model. In this model, the follower’s objective is to identify those hubs the loss of which would most diminish service efficiency. Moreover, the leader’s objective is to identify the set of hubs to locate in order to minimize expected transportation cost while taking normal and failure conditions into account. We have applied two algorithms based on simulated annealing to solve the defined problem. In addition, the algorithms have been calibrated using the Taguchi method. Computational experiments on different instances indicate that the proposed algorithms would be efficient in practice.     

Finite Horizon Minimax Optimal Control of Nonlinear Continuous Time Systems with Stochastic Uncertainty

This paper develops a general continuous-time stochastic framework for robustness analysis and robust control synthesis. We consider a stochastic minimax optimization problem for general stochastic uncertain systems. A general method is presented for converting problems of performance analysis or controller synthesis into unconstrained optimization problems.     

Iterative Path Integral Approach to Nonlinear Stochastic Optimal Control Under Compound Poisson Noise

Nonlinear stochastic optimal control theory has played an important role in many fields. In this theory, uncertainties of dynamics have usually been represented by Brownian motion, which is Gaussian white noise. However, there are many stochastic phenomena whose probability density has a long tail, which suggests the necessity to study the effect of non‐Gaussianity. This paper employs Lévy processes, which cause outliers with a significantly higher probability than Brownian motion, to describe such uncertainties. In general, the optimal control law is obtained by solving the Hamilton–Jacobi–Bellman equation. This paper shows that the path‐integral approach combined with the policy iteration method is efficiently applicable to solve the Hamilton–Jacobi–Bellman equation in the Lévy problem setting. Finally, numerical simulations illustrate the usefulness of this method.