Infinite horizon optimal control for nonlinear interconnected large‐scale dynamical systems with an application to optimal attitude control

This paper presents a novel extended modal series method for solving the infinite horizon optimal control problem of nonlinear interconnected large‐scale dynamic systems. In this method, the infinite horizon nonlinear large‐scale two‐point boundary value problem (TPBVP), derived from Pontryagin's maximum principle, is transformed into a sequence of linear time‐invariant TPBVPs. Solving the latter problems in a recursive manner provides the optimal control law and the optimal trajectory in the form of a uniformly convergent series. Moreover, in special cases, the proposed procedure facilitates the application of parallel processing, which improves its computational efficiency. In this study, an iterative algorithm is also presented, which has a low computational complexity and a fast convergence rate. Just a few iterations are required to obtain a suboptimal trajectory‐control pair. Finally, effectiveness of the proposed approach is verified by solving the optimal attitude control problem. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

Dynamic Games for Stochastic Systems with Delay

This paper discusses dynamic games for a class of linear stochastic delay systems governed by Itô's stochastic differential equation. The Pareto and Nash strategies are developed by solving cross‐coupled matrix inequalities. To obtain these strategy sets, new cross‐coupled algebraic equations (CSAEs) are established on the basis of the Karush‐Kuhn‐Tucker (KKT) conditions, which constitute the necessary conditions. It is noteworthy that the state feedback strategies can be obtained by solving the linear matrix inequality (LMI) recursively. Finally, a numerical example showing the effectiveness of the proposed methods and the attained cost bounds is described.     

Numerical solution of optimal control problems governed by integro‐differential equations

In this paper, an efficient hybrid approximation scheme for solving optimal control problems governed by integro‐differential equations is proposed. The current approach is based on a generalization of the hybrid of block‐pulse functions and Legendre's polynomials. An upper bound for the generalized hybrid functions with respect to the maximum norm is acquired and its convergence is demonstrated. The optimal control problem under study is transcribed to a mathematical programming one. Two illustrative examples are considered to verify the capability and reliability of the proposed procedure.     

基于风险价值约束的动态均值-方差投资组合的研究

研究了基于风险价值约束的动态均值-方差项目投资组合的数学模型,该模型是控制带约束的随机线性二次型（LQ）控制问题.在讨论该随机LQ控制问题的解之后,给出投资组合动态数学模型对应的随机哈密顿-雅克比-贝尔曼方程的解,得出了有效边界和最佳策略,讨论了风险价值约束的影响.最后,针对某油田勘探开发项目的实际情况,应用上述结论求出该实例的解,并讨论了风险价值约束发挥的作用.     

Optimal control for multi-stage and continuous-time linear singular systems

In this paper, optimal control problems for multi-stage and continuous-time linear singular systems are both considered. The singular systems are assumed to be regular and impulse-free. First, a recurrence equation is derived according to Bellman's principle of optimality in dynamic programming. Then, by applying the recurrence equation, bang-bang optimal controls for the control problems with linear objective functions subject to two types of multi-stage singular systems are obtained. Second, employing the principle of optimality, a equation of optimality for settling the optimal control problem subject to a class of continuous-time singular systems is proposed. The optimal control problem may become simpler through solving this equation of optimality. Two numerical examples and a dynamic input–output model are presented to show the effectiveness of the results obtained.     

乘性随机离散系统的最优控制

基于对系统随机不确定因素的分析,文中定义了一种新型随机离散系统--乘性随机 离散系统,并研究该类系统的线性二次型(LQ)最优控制问题.首先给出了该类系统的有限时间 和无限时间LQ最优控制律,并着重分析、证明了无限时间LQ最优控制问题的Riccati方程的 正定矩阵解的存在性及相应数值求解算法与收敛性,以及闭环系统的稳定性等问题.仿真结果 表明了该方法的有效性.     

跳跃扩散股价的最优投资组合选择

假定股票价格服从跳跃扩散过程.在传统均值-方差组合投资模型基础上,最大化最终收益的期望及最小化最终财富的方差.引进一个随机线性二次最优控制问题作为原问题的近似问题.证明了一个状态为跳跃扩散过程的一般最优控制问题的验证性定理.应用验证性定理求解HJB(Hamilton-Jacobi-Bellman)方程得到了原问题的最优策略.最后还给出了原问题有效前沿的表达式.     

含扩散项不可靠生产系统最优生产控制的数值求解

针对含扩散项不可靠随机生产系统最优生产控制的优化命题, 采用数值解方法来求解该优化命题最优控制所满足的模态耦合的非线性偏微分HJB方程. 首先构造Markov链来近似生产系统状态演化, 并基于局部一致性原理, 把求解连续时间随机控制问题转化为求解离散时间的Markov决策过程问题, 然后采用数值迭代和策略迭代算法来实现最优控制数值求解过程. 文末仿真结果验证了该方法的正确性和有效性.     

多阶段均值–方差资产负债管理的随机控制

资产负债管理研究如何合理分配资产以到达最小化风险同时确保期望剩余财富(财富减去负债)达到一定水平.本文在均值–方差投资组合理论的框架下研究两类资产负债管理模型, 包括带有跨期均值–方差投资目标和带有非破产约束的模型. 由于在动态规划意义下, 方差不具有可分性质, 传统的随机最优控制方法难以直接应用. 如采用处理动态均值–方差优化问题的嵌入法来解决以上问题会带来计算上的困难. 本文借鉴平均场控制的思想对以上两类问题加以研究. 本文假设了非常宽泛的市场模型: 所有的资产都是风险资产; 债务和风险资产之间存在相关性. 在此市场假设模型下, 本文给出了最优投资策略(控制率)的解析表达式和均值–方差有效前沿的表达形式. 本研究成果为投资者提供了新的投资策略, 可应用于更复杂的资产负债管理中.     

Iterative Path Integral Approach to Nonlinear Stochastic Optimal Control Under Compound Poisson Noise

Nonlinear stochastic optimal control theory has played an important role in many fields. In this theory, uncertainties of dynamics have usually been represented by Brownian motion, which is Gaussian white noise. However, there are many stochastic phenomena whose probability density has a long tail, which suggests the necessity to study the effect of non‐Gaussianity. This paper employs Lévy processes, which cause outliers with a significantly higher probability than Brownian motion, to describe such uncertainties. In general, the optimal control law is obtained by solving the Hamilton–Jacobi–Bellman equation. This paper shows that the path‐integral approach combined with the policy iteration method is efficiently applicable to solve the Hamilton–Jacobi–Bellman equation in the Lévy problem setting. Finally, numerical simulations illustrate the usefulness of this method.     

Recursive methods for estimating the radial basis function‐based state‐dependent autoregressive model

Identifying a nonlinear radial basis function‐based state‐dependent autoregressive (RBF‐AR) time series model is the basis for solving the corresponding prediction and control problems. This paper studies some recursive parameter estimation algorithms for the RBF‐AR model. Considering the difficulty of the nonlinear optimal problem arising in estimating the RBF‐AR model, an overall forgetting gradient algorithm is deduced based on the negative gradient search. A numerical method with a forgetting factor is provided to solve the problem of determining the optimal convergence factor. In order to improve the parameter estimation accuracy, the multi‐innovation identification theory is applied to develop an overall multi‐innovation forgetting gradient (O‐MIFG) algorithm. The simulation results indicate that the estimation model based on the O‐MIFG algorithm can capture the dynamics of the RBF‐AR model very well.     

A fourth-order B-spline collocation method and its error analysis for Bratu-type and Lane–Emden problems

Recently, Caglar et al. [B-spline method for solving Bratu's problem, Int. J. Comput. Math. 87(8) (2010), pp. 1885–1891] proposed a numerical technique based on cubic B-spline for solving a Bratu-type problem. This method provides a second-order convergent approximation to the solution of the problem. In this paper, we develop a high-order numerical method based on quartic B-spline collocation approach for the Bratu-type and Lane–Emden problems. The error analysis of the quartic B-spline interpolation is carried out. Some numerical examples are provided to demonstrate the efficiency and applicability of the method and to verify its rate of convergence. The numerical results are compared with exact solutions and a numerical method based on cubic B-spline approach. Comparison reveals that our method produces more accurate results than the method proposed by Caglar et al. [B-spline method for solving Bratu's problem, Int. J. Comput. Math. 87(8) (2010), pp. 1885–1891].     

Distributed impulsive synchronization of Lur'e dynamical networks via parameter variation methods

This paper is devoted to investigating the exponential synchronization of coupled Lur'e dynamical networks with multiple time‐varying delays and stochastic disturbance. The problem studied in this paper could be regarded as a kind of leader‐following synchronization issue. As the networks may suffer from certain impulsive disturbance, an effective distributed impulsive control protocol is proposed to synchronize the stochastic Lur'e dynamical networks. According to the comparison principle, the average impulsive interval, and the extended formula for the variation of parameters, sufficient conditions are derived for successful achievement of the network synchronization with consideration to different functions of impulsive effects. Furthermore, the exponential convergence rate is obtained based on the impulsive solution equation. In addition, finally, some numerical simulations are given to illustrate the validity of the control scheme and the theoretical analysis.     

On the infinite time solution to state-constrained stochastic optimal control problems

A method is presented for solving the infinite time Hamilton-Jacobi-Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution.     

Aircraft Trajectory Optimization for Collision Avoidance Using Stochastic Optimal Control

Optimizing aircraft collision avoidance and performing trajectory optimization are the key problems in an air transportation system. This paper is focused on solving these problems by using a stochastic optimal control approach. The major contribution of this paper is a proposed stochastic optimal control algorithm to dynamically adjust and optimize aircraft trajectory. In addition, this algorithm accounts for random wind dynamics and convective weather areas with changing size. Although the system is modeled by a stochastic differential equation, the optimal feedback control for this equation can be computed as a solution of a partial differential equation, namely, an elliptic Hamilton‐Jacobi‐Bellman equation. In this paper, we solve this equation numerically using a Markov Chain approximation approach, where a comparison of three different iterative methods and two different optimization search methods are presented. Simulations show that the proposed method provides better performance in reducing conflict probability in the system and that it is feasible for real applications.     

Numerical solution of continuous-time mean–variance portfolio selection with nonlinear constraints

An investment problem is considered with dynamic mean–variance (M–V) portfolio criterion under discontinuous prices described by jump-diffusion processes. Some investment strategies are restricted in the study. This M–V portfolio with restrictions can lead to a stochastic optimal control model. The corresponding stochastic Hamilton–Jacobi–Bellman equation of the problem with linear and nonlinear constraints is derived. Numerical algorithms are presented for finding the optimal solution in this article. Finally, a computational experiment is to illustrate the proposed methods by comparing with M–V portfolio problem which does not have any constraints.     

Multiscale methods for the valuation of American options with stochastic volatility

This paper deals with the efficient valuation of American options. We adopt Heston's approach for a model of stochastic volatility, leading to a generalized Black–Scholes equation called Heston's equation. Together with appropriate boundary conditions, this can be formulated as a parabolic boundary value problem with a free boundary, the optimal exercise price of the option. For its efficient numerical solution, we employ, among other multiscale methods, a monotone multigrid method based on linear finite elements in space and display corresponding numerical experiments.     

Optimal portfolio trading subject to stochastic dominance constraints under second‐order autoregressive price dynamics

This paper studies the optimal portfolio trading problem under the generalized second‐order autoregressive execution price model. The problem of minimizing expected execution cost under the proposed price model is formulated as a quadratic programming (QP) problem. For a risk‐averse trader, problem formulation under the second‐order stochastic dominance constraints results in a quadratically constrained QP problem. Under some conditions on the execution price model, it is proved that the portfolio trading problems for risk‐neutral and risk‐averse traders become convex programming problems, which have many theoretical and computational advantages over the general class of optimization problems. Extensive numerical illustrations are provided, which render the practical significance of the proposed execution price model and the portfolio trading problems.     

Distributed and boundary optimal control of the Allen–Cahn equation with regular potential and dynamic boundary conditions

This paper is concerned with an optimal control problem (P) (both distributed control as well as boundary control) for the nonlinear phase-field (Allen–Cahn) equation, involving a regular potential and dynamic boundary condition. A family of approximate optimal control problems (P^?) is introduced and results for the existence of an optimal control for problems (P) and (P^?) are proven. Furthermore, the convergence result of the optimal solution of problem (P^?) to the optimal solution of problem (P) is proved. Besides the existence of an optimal control in problem (P^?), necessary optimality conditions (Pontryagin's principle) as well as a conceptual gradient-type algorithm to approximate the optimal control, were established in the end.     

A numerical solution of the stochastic discrete algebraic Riccati equation

This article proposes two algorithms for solving a stochastic discrete algebraic Riccati equation which arises in a stochastic optimal control problem for a discrete-time system. Our algorithms are generalized versions of Hewer’s algorithm. Algorithm I has quadratic convergence, but needs to solve a sequence of extended Lyapunov equations. On the other hand, Algorithm II only needs solutions of standard Lyapunov equations which can be solved easily, but it has a linear convergence. By a numerical example, we shall show that Algorithm I is superior to Algorithm II in cases of large dimensions. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008