A survey of feedback particle filter and related controlled interacting particle systems (CIPS)

In this survey, we describe controlled interacting particle systems (CIPS) to approximate the solution of the optimal filtering and the optimal control problems. Part I of the survey is focussed on the feedback particle filter (FPF) algorithm, its derivation based on optimal transportation theory, and its relationship to the ensemble Kalman filter (EnKF) and the conventional sequential importance sampling–resampling (SIR) particle filters. The central numerical problem of FPF—to approximate the solution of the Poisson equation—is described together with the main solution approaches. An analytical and numerical comparison with the SIR particle filter is given to illustrate the advantages of the CIPS approach. Part II of the survey is focussed on adapting these algorithms for the problem of reinforcement learning. The survey includes several remarks that describe extensions as well as open problems in this subject.     

Optimal control problems with multiple characteristic time points in the objective and constraints

In this paper, we develop a computational method for a class of optimal control problems where the objective and constraint functionals depend on two or more discrete time points. These time points can be either fixed or variable. Using the control parametrization technique and a time scaling transformation, this type of optimal control problem is approximated by a sequence of approximate optimal parameter selection problems. Each of these approximate problems can be viewed as a finite dimensional optimization problem. New gradient formulae for the cost and constraint functions are derived. With these gradient formulae, standard gradient-based optimization methods can be applied to solve each approximate optimal parameter selection problem. For illustration, two numerical examples are solved.     

Stabilization of sampled-data nonlinear systems by receding horizon control via discrete-time approximations

Results on stabilizing receding horizon control of sampled-data nonlinear systems via their approximate discrete-time models are presented. The proposed receding horizon control is based on the solution of Bolza-type optimal control problems for the parametrized family of approximate discrete-time models. This paper investigates both situations when the sampling period T is fixed and the integration parameter h used in obtaining approximate model can be chosen arbitrarily small, and when these two parameters coincide but they can be adjusted arbitrary. Sufficient conditions are established which guarantee that the controller that renders the origin to be asymptotically stable for the approximate model also stabilizes the exact discrete-time model for sufficiently small integration and/or sampling parameters.     

Investigation of Commutative Properties of Discontinuous Galerkin Methods in PDE Constrained Optimal Control Problems

The aim of this paper is to investigate commutative properties of a large family of discontinuous Galerkin (DG) methods applied to optimal control problems governed by the advection-diffusion equations. To compute numerical solutions of PDE constrained optimal control problems there are two main approaches: optimize-then-discretize and discretize-then-optimize. These two approaches do not always coincide and may lead to substantially different numerical solutions. The methods for which these two approaches do coincide we call commutative. In the theory of single equations, there is a related notion of adjoint or dual consistency. In this paper we classify DG methods both in primary and mixed forms and derive necessary conditions that can be used to develop new commutative methods. We will also derive error estimates in the energy and L ² norms. Numerical examples reveal that in the context of PDE constrained optimal control problems a special care needs to be taken to compute the solutions. For example, choosing non-commutative methods and discretize-then-optimize approach may result in a badly behaved numerical solution.     

Numerical Solution of Optimal Control Problems with Constant Control Delays

We investigate a class of optimal control problems that exhibit constant exogenously given delays in the control in the equation of motion of the differential states. Therefore, we formulate an exemplary optimal control problem with one stock and one control variable and review some analytic properties of an optimal solution. However, analytical considerations are quite limited in case of delayed optimal control problems. In order to overcome these limits, we reformulate the problem and apply direct numerical methods to calculate approximate solutions that give a better understanding of this class of optimization problems. In particular, we present two possibilities to reformulate the delayed optimal control problem into an instantaneous optimal control problem and show how these can be solved numerically with a state-of-the-art direct method by applying Bock’s direct multiple shooting algorithm. We further demonstrate the strength of our approach by two economic examples.      

The sample solution approach for determination of the optimal shape parameter in the Multiquadric function of the Kansa method

The Kansa method with the Multiquadric-radial basis function (MQ-RBF) is inherently meshfree and can achieve an exponential convergence rate if the optimal shape parameter is available. However, it is not an easy task to obtain the optimal shape parameter for complex problems whose analytical solution is often a priori unknown. This has long been a bottleneck for the MQ-Kansa method application to practical problems. In this paper, we present a novel sample solution approach (SSA) for achieving a reasonably good shape parameter of the MQ-RBF in the Kansa method for the solution of problems whose analytical solution is unknown. The basic assumption behind the SSA is that the optimal shape parameter is considered to be largely depended on the shape of computational domain, the type of the boundary conditions, the number and distribution of nodes, and the governing equation. In the procedure of the SSA, we set up a pseudo-problem as the sample solution whose solution is known. It is not difficult to obtain the optimal parameter of the MQ-RBF in the numerical solution of the pseudo-problem. The SSA suggests that the optimal shape parameter of the pseudo-problem can also achieve an approximately optimal accuracy in the solution of the original problem. Numerical examples and comparisons are provided to verify the proposed SSA in terms of accuracy and stability in solving homogeneous problems and non-homogeneous modified Helmholtz problems in several complex domains even using chaotic distribution of collocation points.     

Active Rocket Trajectory Arcs: A Review

This paper reviews the development of analytical, approximate analytical, and numerical methods for solving the variational problem on the determination of optimal rocket trajectories in gravitational fields, and their application to study flight dynamics. Specifics of these methods as applied to solve modern and complex problems are described. A variational problem is formulated and extremal thrust arcs are described. Papers containing results of analytical investigations on thrust arcs are reviewed in depth. Partially investigated problems are described. Problems of great interest in the development of methods for solving the variational problem and problems in the theory of optimal trajectories are mentioned.     

Convergence analysis of a computational method for time-lag optimal control problems

We consider a class of non-linear time-lag optimal control problems. The class of admissible controls are taken to be the class of piecewise smooth functions. A control parameterization technique is used to approximate the optimal control problem by a sequence of optimal parameter selection problems. The solution of each of these approximate problems gives rise to a sub-optimal solution to the true optimal control problem in an obvious way. The error bound is derived for the sub-optimal costs and the true optimal cost.     

A penalty method for history-dependent variational–hemivariational inequalities

Penalty methods approximate a constrained variational or hemivariational inequality problem through a sequence of unconstrained ones as the penalty parameter approaches zero. The methods are useful in the numerical solution of constrained problems, and they are also useful as a tool in proving solution existence of constrained problems. This paper is devoted to a theoretical analysis of penalty methods for a general class of variational–hemivariational inequalities with history-dependent operators. Unique solvability of penalized problems is shown, as well as the convergence of their solutions to the solution of the original history-dependent variational–hemivariational inequality as the penalty parameter tends to zero. The convergence result proved here generalizes several existing convergence results of penalty methods. Finally, the theoretical results are applied to examples of history-dependent variational–hemivariational inequalities in mathematical models describing the quasistatic contact between a viscoelastic rod and a reactive foundation.     

High-order scheme for determination of a control parameter in an inverse problem from the over-specified data

The problem of finding the solution of partial differential equations with source control parameter has appeared increasingly in physical phenomena, for example, in the study of heat conduction process, thermo-elasticity, chemical diffusion and control theory. In this paper we present a high order scheme for determining unknown control parameter and unknown solution of parabolic inverse problem with both integral overspecialization and overspecialization at a point in the spatial domain. In these equations, we first approximate the spatial derivative with a fourth order compact scheme and reduce the problem to a system of ordinary differential equations (ODEs). Then we apply a fourth order boundary value method for the solution of resulting system of ODEs. So the proposed method has fourth order accuracy in both space and time components and is unconditionally stable due to the favorable stability property of boundary value methods. Several numerical examples and also some comparisons with other methods in the literature will be investigated to confirm the efficiency of the new procedure.     

Convergence analysis of a computational method for optimal control of non-linear differential-algebraic systems

The convergence analysis of a computational method for optimal control problems of non-linear differential-algebraic systems is considered. The class of admissible controls is taken to be the class of piecewise smooth functions. A control parametrizution technique is used to approximate the optimal control problem into a sequence of optimal parameter selection problems. The solution of each of these approximate problems gives rise to a suboptimal solution to the original optimal control problem in an obvious way. The gradients of the cost functional with respect to parameters are derived. Furthermore, the error bounds between the suboptimal costs and the true optimal cost are derived.     

Pontryagin’s maximum principle in optimal control problems of orientation of a spacecraft

Pontryagin’s maximum principle is used for solution of topical problems of spacecraft motion control. The dynamic optimal control problem of space orientation of a spacecraft from an arbitrary initial to a given final angular position with minimization of the turning time is studied in detail. The solution to the formulated problem is obtained and numerical expressions for synthesis of optimal control program are given. Results of mathematical simulation of the dynamics of motion of a spacecraft at optimal control are presented; these results demonstrate practical feasibility of the developed control algorithm.     

近似强化学习算法研究综述

强化学习用于解决无模型情况下的优化决策问题,是实现人工智能的重要技术之一,但传统的表格型强化学习方法难以处理具有大规模、连续空间的控制问题。近似强化学习受到函数逼近思想的启发,对价值函数或策略函数参数化表示,通过参数优化间接获得最优行为策略,在视频游戏、棋类对抗及机器人控制等领域应用效果显著。基于此,对近似强化学习算法的研究现状与应用进展进行了梳理和综述。介绍了近似强化学习相关的基础理论;分类总结了近似强化学习的经典算法及一些相应的改进方法;概述了近似强化学习在机器人控制领域的研究进展,并总结了当前面临的若干主要问题,为后续的研究提供参考。     

非线性动态系统的容错控制

首先概要介绍了非线性动态系统容错控制技术的发展现状;然后分类介绍了几种非线性系统的容错控制技术,重点分析了基于人工智能和参数估计的主动容错控制方法和基于Hamilton-Jacobi方程的非线性被动容错控制设计方法。对于其它的方法,则简要讨论了它们的适用范围和优缺点;并探讨了该领域的难点问题和可能的研究方向。     

Exact and approximate Riemann solvers for compressible two-phase flows

Numerical methods for solving equations of two-phase hydrodynamics, which describe the flow of a dispersed solid and gas mixture are considered. The Godunov method is applied as the main approach to approximate numerical fluxes in solutions of the relevant Riemann problems. The formulations of these problems for the solid and gas phases are given, their exact analytical solution is described, and possible simplified approximate solutions are discussed. The obtained theoretical results are applied to the construction of a discrete model, which results in the generalization of the well-known Godunov-type and Rusanov-type methods to the case of nonequilibrium two-phase media. The numerical results involve the verification of the constructed methods on the analytical solutions of two-phase equations.     

Solving Initial Value Problems for Ordinary Differential Equations by two approaches: BDF and piecewise-linearized methods

Many scientific and engineering problems are described using Ordinary Differential Equations (ODEs), where the analytic solution is unknown. Much research has been done by the scientific community on developing numerical methods which can provide an approximate solution of the original ODE. In this work, two approaches have been considered based on BDF and Piecewise-linearized Methods. The approach based on BDF methods uses a Chord-Shamanskii iteration for computing the nonlinear system which is obtained when the BDF schema is used. Two approaches based on piecewise-linearized methods have also been considered. These approaches are based on a theorem proved in this paper which allows to compute the approximate solution at each time step by means of a block-oriented method based on diagonal Padé approximations. The difference between these implementations is in using or not using the scale and squaring technique.Five algorithms based on these approaches have been developed. MATLAB and Fortran versions of the above algorithms have been developed, comparing both precision and computational costs. BLAS and LAPACK libraries have been used in Fortran implementations. In order to compare in equality of conditions all implementations, algorithms with fixed step have been considered. Four of the five case studies analyzed come from biology and chemical kinetics stiff problems. Experimental results show the advantages of the proposed algorithms, especially when they are integrating stiff problems.     

Numerical solution of some initial optimal control problems using the reproducing kernel Hilbert space technique

ABSTRACT

In this paper, we present a general technique for solving a class of linear/nonlinear optimal control problems. In fact, an analytical solution of the state variable is represented in the form of a series in a reproducing kernel Hilbert space. Sometimes with the aid of this series form, we can also present the optimal control variable in a series form. An iterative method is given to obtain the approximate optimal control and state variables and the cost functional is numerically obtained. Convergence analysis of the method is also provided. Several numerical examples are tested to demonstrate the applicability and efficiency of the method.     

Stochastic Control of Linear and Nonlinear Econometric Models: Some Computational Aspects

This paper considers the optimal control of small econometric models applying the OPTCON algorithm. OPTCON determines approximate numerical solutions to optimum control problems for nonlinear stochastic systems. These optimum control problems consist in minimizing a quadratic objective function for linear and nonlinear econometric models with additive and multiplicative (parameter) uncertainties. The algorithm was programmed in C# and in MATLAB and allows for stochastic control with open-loop and passive learning (open-loop feedback) information patterns. Here we compare the results of applying the OPTCON2 version of the algorithm to two macroeconomic models for Slovenia, the nonlinear model SLOVNL and the linear model SLOVL. The results for both models are similar, with open-loop feedback controls giving better results on average and less outliers than open-loop controls. The number of outliers is higher in the nonlinear case and especially under high parameter uncertainty.     

含时滞的半主动系统动力学分析

对含时滞的半主动相对控制悬架系统进行了近似解析研究.首先建立了半主动相对控制1/4车体模型,进行了无量纲化处理,利用平均法建立了系统的近似解析解应该满足的四元代数方程组,然后利用数值方法进行了求解.随后通过MATLAB仿真得到了含时滞的半主动相对控制悬架系统的数值解,并且和近似解析解进行了比较,发现二者具有较好的符合精度,说明近似解析解的正确性.     

Nonlinear singularly perturbed optimal control problems with singular arcs

A third-order, nonlinear, singularly perturbed optimal control problem is considered under assumptions which assure that the full problem is singular and the reduced problem is nonsingular. The separation between the singular arc of the full problem and the optimal control law of the reduced one, both of which are hypersurfaces in state space, is of the same order as the small parameter of the problem. Boundary-layer solutions are constructed which are stable and reach the outer solution in a finite time. A uniformly valid composite solution is then formed from the reduced and boundary-layer solutions. The value of the approximate solution is that it is relatively easy to obtain and does not involve singular arcs. To illustrate the utility of the results, the technique is used to obtain an approximate solution of a simplified version of the aircraft minimum time-to-climb problem. A numerical example is included.