Maximum principle for forward–backward SDEs with a general cost functional

In this study, we consider a stochastic optimal control problem whose state variables are described by the system of forward and backward stochastic differential equations with a general cost functional which relies on the global terminal condition. In the framework of Fréchet derivatives, we derive the corresponding maximum principle via constructing a series of adjoint equations which need to be solved step by step.     

Solvability of indefinite stochastic Riccati equations and linear quadratic optimal control problems

A new approach to study the indefinite stochastic linear quadratic (LQ) optimal control problems, which we called the “equivalent cost functional method”, is introduced by Yu (2013) in the setup of Hamiltonian system. On the other hand, another important issue along this research direction, is the possible state feedback representation of optimal control and the solvability of associated indefinite stochastic Riccati equations. As the response, this paper continues to develop the equivalent cost functional method by extending it to the Riccati equation setup. Our analysis is featured by its introduction of some equivalent cost functionals which enable us to have the bridge between the indefinite and positive-definite stochastic LQ problems. With such bridge, some solvability relation between the indefinite and positive-definite Riccati equations is further characterized. It is remarkable the solvability of the former is rather complicated than the latter, hence our relation provides some alternative but useful viewpoint. Consequently, the corresponding indefinite linear quadratic problem is discussed for which the unique optimal control is derived in terms of state feedback via the solution of the Riccati equation. In addition, some example is studied using our theoretical results.     

The mean squared loss control problem for a partially observed Markov chain

In this article, we study the optimal control of a partially observed Markov chain for which a mean squared cost functional is minimised. Both the terminal cost and the running cost are considered. Minimum principles are established. In both cases, if the optimal control is Markov feedback, more explicit forms for the stochastic integrands and adjoint processes are obtained.     

Pollution's ambient problems and regularity of optimal cost function

We study pollution's ambient problems by using the optimal control theory applied to partial differential equations. We consider the problem to find the optimal way to eliminate pollution in the time, such that the concentration is close to a standard level which does not affect the ecological equilibrium when the source is pointwise. We consider a quadratic cost functional and we prove the existence and uniqueness of optimal control. We find a characterisation which makes possible the computing of optimal control. Additionally, we consider the problem moving the pointwise source. So, we define a function j(b) that associates to any point b ∈ Ω the optimal cost functional applied to the optimal control. We show that j is differentiable, provided that the controls are taken in a convenient subset of admissible functions satisfying the cone properties. We also find the point in Ω, for which the cost functional is minimum.     

Computation of the Hessian matrix in aerodynamic inverse design using continuous adjoint formulations

Continuous adjoint formulations for the computation of (first and) second order derivatives of the objective function governing inverse design problems in 2D inviscid flows are presented. These are prerequisites for the use of the very efficient exact Newton method. Four new formulations based on all possible combinations of the direct differentiation method and the continuous adjoint approach to compute the sensitivity derivatives of objective functions, constrained by the flow equations, are presented. They are compared in terms of the expected CPU cost to compute the Hessian of the objective function used in single-objective optimization problems with N degrees of freedom. The less costly among them was selected for further study and tested in inverse design problems solved by means of the Newton method. The selected approach, which will be referred to as the direct-adjoint one, since it performs direct differentiation for the gradient and, then, uses the adjoint approach to compute the Hessian, requires as many as N+2 equivalent flow solutions for each Newton step. The major part of the CPU cost (N equivalent flow solutions) is for the computation of the gradient but, fortunately, this task is directly amenable to parallelization. The method is used to reconstruct ducts or cascade airfoils for a known pressure distribution along their solid boundaries, at inviscid flow conditions. The examined cases aim at demonstrating the accuracy of the proposed method in computing the exact Hessian matrix as well as the efficiency of the exact Newton method as an optimization tool in aerodynamic design.     

Certified Reduced Basis Methods for Parametrized Elliptic Optimal Control Problems with Distributed Controls

In this paper, we consider the efficient and reliable solution of distributed optimal control problems governed by parametrized elliptic partial differential equations. The reduced basis method is used as a low-dimensional surrogate model to solve the optimal control problem. To this end, we introduce reduced basis spaces not only for the state and adjoint variable but also for the distributed control variable. We also propose two different error estimation procedures that provide rigorous bounds for the error in the optimal control and the associated cost functional. The reduced basis optimal control problem and associated a posteriori error bounds can be efficiently evaluated in an offline–online computational procedure, thus making our approach relevant in the many-query or real-time context. We compare our bounds with a previously proposed bound based on the Banach–Ne?as–Babu?ka theory and present numerical results for two model problems: a Graetz flow problem and a heat transfer problem. Finally, we also apply and test the performance of our newly proposed bound on a hyperthermia treatment planning problem.     

Optimal aerodynamic design of airfoils in unsteady viscous flows

A continuous adjoint formulation is used to determine optimal airfoil shapes in unsteady viscous flows at Re = 1 × 10⁴. The Reynolds number is based on the free-stream speed and the chord length of the airfoil. A finite element method based on streamline-upwind Petrov/Galerkin (SUPG) and pressure-stabilized Petrov/Galerkin (PSPG) stabilizations is used to solve both the flow and adjoint equations. The airfoil is parametrized via a Non-Uniform Rational B-Splines (NURBS) curve. Three different objective functions are used to obtain optimal shapes: maximize lift, minimize drag and minimize ratio of drag to lift. The objective functions are formulated on the basis of time-averaged aerodynamic coefficients. The three objective functions result in diverse airfoil geometries. The resulting airfoils are thin, with the largest thickness to chord ratio being only 5.4%. The shapes obtained are further investigated for their aerodynamic performance. Maximization of time-averaged lift leads to an airfoil that produces more than six times more lift compared to the NACA 0012 airfoil. The excess lift is a consequence of the large peak and extended region of high suction on the upper surface and high pressure on the lower surface. Minimization of drag results in an airfoil with a sharp leading edge. The flow remains attached for close to 70% of the chord length. Minimization of the ratio of drag to lift results in an airfoil with a shallow dimple on the upper surface. It leads to a fairly large value of the time-averaged ratio of lift to drag (～ 17.8). The high value is mostly achieved by a 447% increase in lift and 16% reduction in drag, compared to a NACA 0012 airfoil. Imposition of volume constraint, for the cases studied, is found to result in airfoils that have lower aerodynamic performance.     

Suboptimal control of linear systems with delays in state and input by orthonormal basis

This paper presents a method for finding the optimal control of linear systems with delays in state and input and the quadratic cost functional using an orthonormal basis for square integrable functions. The state and control variables and their delays are expanded in an orthonormal basis with unknown coefficients. Using operational matrices of integration, delay and product, the relation between coefficients of variables is provided. Then, necessary condition of optimality is driven as a linear system of algebraic equations in terms of the unknown coefficients of state and control variables. As an application of this method, the linear Legendre multiwavelets as an orthonormal basis for L ²[0, 1] are used. Two time-delayed optimal control problems are approximated to establish the usefulness of the method.     

Necessary and sufficient conditions of risk‐sensitive optimal control and differential games for stochastic differential delayed equations

In this paper, we consider risk‐sensitive optimal control and differential games for stochastic differential delayed equations driven by Brownian motion. The problems are related to robust stochastic optimization with delay due to the inherent feature of the risk‐sensitive objective functional. For both problems, by using the logarithmic transformation of the associated risk‐neutral problem, the necessary and sufficient conditions for the risk‐sensitive maximum principle are obtained. We show that these conditions are characterized in terms of the variational inequality and the coupled anticipated backward stochastic differential equations (ABSDEs). The coupled ABSDEs consist of the first‐order adjoint equation and an additional scalar ABSDE, where the latter is induced due to the nonsmooth nonlinear transformation of the adjoint process of the associated risk‐neutral problem. For applications, we consider the risk‐sensitive linear‐quadratic control and game problems with delay, and the optimal consumption and production game, for which we obtain explicit optimal solutions.     

非线性离散系统的近似最优跟踪控制

研究非线性离散系统的最优跟踪控制问题. 通过在由最优控制问题所导致的非线性两点边值问题中引入灵敏度参数, 并对它进行Maclaurin级数展开, 将原最优跟踪控制问题转化为一族非齐次线性两点边值问题. 得到的最优跟踪控制由解析的前馈反馈项和级数形式的补偿项组成. 解析的前馈反馈项可以由求解一个Riccati差分方程和一个矩阵差分方程得到. 级数补偿项可以由一个求解伴随向量的迭代算法近似求得. 以连续槽式反应器为例进行仿真验证了该方法的有效性．     

Aerodynamic design via control theory

Conclusion The purpose of the last three sections is to demonstrate by representative examples that control theory can be used to formulate computationally feasible procedures for aerodynamic design. The cost of each iteration is of the same order as two flow solutions, since the adjoint equation is of comparable complexity to the flow equation, and the remaining auxiliary equations could be solved quite inexpensively. Provided, therefore, that one can afford the cost of a moderate number of flow solutions, procedures of this type can be used to derive improved designs. The approach is quite general, not limited to particular choices of the coordinate transformation or cost function, which might in fact contain measures of other criteria of performance such as lift and drag. For the sake of simplicity certain complicating factors, such as the need to include a special term in the mapping function to generate a corner at the trailing edge, have been suppressed from the present analysis. Also it remains to explore the numerical implementation of the design procedures proposed in this paper.     

A contact force model considering constant external forces for impact analysis in multibody dynamics

The adjoint method is an elegant approach for the computation of the gradient of a cost function to identify a set of parameters. An additional set of differential equations has to be solved to compute the adjoint variables, which are further used for the gradient computation. However, the accuracy of the numerical solution of the adjoint differential equation has a great impact on the gradient. Hence, an alternative approach is the discrete adjoint method, where the adjoint differential equations are replaced by algebraic equations. Therefore, a finite difference scheme is constructed for the adjoint system directly from the numerical time integration method. The method provides the exact gradient of the discretized cost function subjected to the discretized equations of motion.     

Solution of the input-constrained LQR problem using dynamic programming

The input-constrained LQR problem is addressed in this paper; i.e., the problem of finding the optimal control law for a linear system such that a quadratic cost functional is minimised over a horizon of length N subject to the satisfaction of input constraints. A global solution (i.e., valid in the entire state space) for this problem, and for arbitrary horizon N, is derived analytically by using dynamic programming. The scalar input case is considered in this paper. Solutions to this problem (and to more general problems: state constraints, multiple inputs) have been reported recently in the literature, for example, approaches that use the geometric structure of the underlying quadratic programming problem and approaches that use multi-parametric quadratic programming techniques. The solution by dynamic programming proposed in the present paper coincides with the ones obtained by the aforementioned approaches. However, being derived using a different approach that exploits the dynamic nature of the constrained optimisation problem to obtain an analytical solution, the present result complements the previous methods and reveals additional insights into the intrinsic structure of the optimal solution.     

Optimal control of markov chains admitting strong and weak interactions

This paper is devoted to the study of an optimal control problem for a Markov chain with generator B + εA, where ε is a small parameter. It is shown that an approximate solution can be calculated by a policy improvement algorithm involving computations relative to an ‘aggregated’ problem (the dimension of which is given by N, the number of ergodic sets for the B matrix) together with a family of ‘decentralized’ problems (the dimensions of which are given by the number of elements in each ergodic set for the B matrix).     

Uniting local and global controllers for uncertain nonlinear systems: beyond global inverse optimality

This paper provides a solution to a new problem of global robust control for uncertain nonlinear systems. A new recursive design of stabilizing feedback control is proposed in which inverse optimality is achieved globally through the selection of generalized state-dependent scaling. The inverse optimal control law can always be designed such that its linearization is identical to linear optimal control, i.e. optimal control, for the linearized system with respect to a prescribed quadratic cost functional. Like other backstepping methods, this design is always successful for systems in strict-feedback form. The significance of the result stems from the fact that our controllers achieve desired level of ‘global’ robustness which is prescribed a priori. By uniting locally optimal robust control and global robust control with global inverse optimality, one can obtain global control laws with reasonable robustness without solving Hamilton–Jacobi equations directly.     

聚合物驱最优控制问题求解算法的设计与实现

为了获得聚合物驱油的最大利润,通过最优控制来确定聚合物的最佳注入策略是一种有效的方法。该最优控制问题的数值解涉及到油藏数值模拟、伴随方程和非线性规划问题。给出了基于面向对象的算法设计方案及其实现细节。利用全隐式差分格式离散化聚合物驱模型,并采用Newton-Raphson求解所得到非线性方程组,在求解前向模型的同时构造了伴随方程。对一个三维聚合物驱注入问题进行了实例求解,表明了所实现算法的实用性和有效性。     

The maximum principle for stochastic differential systems with general cost functional

In this paper, under the framework of Fréchet derivatives, we study a stochastic optimal control problem driven by a stochastic differential equation with general cost functional. By constructing a series of first-order and second-order adjoint equations, we establish the stochastic maximum principle and get the related Hamilton systems.     

On solving hybrid optimal control problems with higher index DAEs

The paper deals with hybrid optimal control problems described by higher index differential–algebraic equations (DAEs). We introduce a numerical procedure for solving these problems. The procedure has the following features: it is based on the appropriately defined adjoint equations formulated for the discretized equations being the result of the numerical integration of systems equations by an implicit Runge–Kutta method; the consistent initialization procedure is applied whenever control functions jumps, or state variables transition occurs. The procedure can cope with hybrid optimal control problems which are defined by DAEs with the index not exceeding three. Our approach does not require differentiation of some system equations in order to transform higher index DAEs to the underlying ordinary differential equations (ODEs). The presented numerical examples show that the proposed approach can be used to solve efficiently hybrid optimal control problems with higher index DAEs.     

Intelligent mixed H2/H∞ adaptive tracking control system design using self-organizing recurrent fuzzy-wavelet-neural-network for uncertain two-axis motion control system

In this paper, an intelligent adaptive tracking control system (IATCS) based on the mixed H₂/H_∞ approach under uncertain plant parameters and external disturbances for achieving high precision performance of a two-axis motion control system is proposed. The two-axis motion control system is an X–Y table driven by two permanent-magnet linear synchronous motors (PMLSMs) servo drives. The proposed control scheme incorporates a mixed H₂/H_∞ controller, a self-organizing recurrent fuzzy-wavelet-neural-network controller (SORFWNNC) and a robust controller. The combinations of these control methods would insure the stability, robustness, optimality, overcome the uncertainties, and performance properties of the two-axis motion control system. The SORFWNNC is used as the main tracking controller to adaptively estimate an unknown nonlinear dynamic function that includes the lumped parameter uncertainties, external disturbances, cross-coupled interference and frictional force. Moreover, the structure and the parameter learning phases of the SORFWNNC are performed concurrently and online. Furthermore, a robust controller is designed to deal with the uncertainties, including the approximation error, optimal parameter vectors and higher order terms in Taylor series. Besides, the mixed H₂/H_∞ controller is designed such that the quadratic cost function is minimized and the worst case effect of the unknown nonlinear dynamic function on the tracking error must be attenuated below a desired attenuation level. The mixed H₂/H_∞ control design has the advantage of both H₂ optimal control performance and H_∞ robust control performance. The sufficient conditions are developed for the adaptive mixed H₂/H_∞ tracking problem in terms of a pair of coupled algebraic equations instead of coupled nonlinear differential equations. The coupled algebraic equations can be solved analytically. The online adaptive control laws are derived based on Lyapunov theorem and the mixed H₂/H_∞ tracking performance so that the stability of the proposed IATCS can be guaranteed. Furthermore, the control algorithms are implemented in a DSP-based control computer. From the experimental results, the motions at X-axis and Y-axis are controlled separately, and the dynamic behaviors of the proposed IATCS can achieve favorable tracking performance and are robust to parameter uncertainties.