Convergence of optimal control problems governed by second kind parabolic variational inequalities

We consider a family of optimal control problems where the control variable is given by a boundary condition of Neumann type. This family is governed by parabolic variational inequalities of the second kind. We prove the strong convergence of the optimal control and state systems associated to this family to a similar optimal control problem. This work solves the open problem left by the authors in IFIP TC7 CSMO2011.     

On the Hybrid Optimal Control Problem: Theory and Algorithms

A class of hybrid optimal control problems (HOCP) for systems with controlled and autonomous location transitions is formulated and a set of necessary conditions for hybrid system trajectory optimality is presented which together constitute generalizations of the standard Maximum Principle; these are given for the cases of open bounded control value sets and compact control value sets. The derivations in the paper employ: (i) classical variational and needle variation techniques; and (ii) a local controllability condition which is used to establish the adjoint and Hamiltonian jump conditions in the autonomous switching case. Employing the hybrid minimum principle (HMP) necessary conditions, a class of general HMP based algorithms for hybrid systems optimization are presented and analyzed for the autonomous switchings case and the controlled switchings case. Using results from the theory of penalty function methods and Ekeland's variational principle the convergence of these algorithms is established under reasonable assumptions. The efficacy of the proposed algorithms is illustrated via computational examples.     

Contraction analysis of non-linear distributed systems

Contraction theory is a comparatively recent dynamic analysis and non-linear control system design tool based on an exact differential analysis of convergence. This paper extends contraction theory to local and global stability analysis of important classes of non-linear distributed dynamics, such as convection-diffusion-reaction processes, Lagrangian and Hamilton–Jacobi dynamics, and optimal controllers and observers. By contrast with stability proofs based on energy dissipation, stability and convergence can be determined for energy-based systems excited by time-varying inputs.

The Hamilton–Jacobi–Bellman controller and a similar optimal non-linear observer design are studied based on explicitly computable conditions on the convexity of the cost function. These stability conditions extend the well-known conditions on controllability and observability Grammians for linear time-varying systems, without requiring the unknown transition matrix of the underlying differential dynamics.     

A Computational Method for Stochastic Optimal Control Problems in Financial Mathematics

Principle of optimality or dynamic programming leads to derivation of a partial differential equation (PDE) for solving optimal control problems, namely the Hamilton‐Jacobi‐Bellman (HJB) equation. In general, this equation cannot be solved analytically; thus many computing strategies have been developed for optimal control problems. Many problems in financial mathematics involve the solution of stochastic optimal control (SOC) problems. In this work, the variational iteration method (VIM) is applied for solving SOC problems. In fact, solutions for the value function and the corresponding optimal strategies are obtained numerically. We solve a stochastic linear regulator problem to investigate the applicability and simplicity of the presented method and prove its convergence. In particular, for Merton's portfolio selection model as a problem of portfolio optimization, the proposed numerical method is applied for the first time and its usefulness is demonstrated. For the nonlinear case, we investigate its convergence using Banach's fixed point theorem. The numerical results confirm the simplicity and efficiency of our method.     

A general one-dimensional search algorithm for computing approximate solutions of some non-linear optimal control problems

Barr and Gilbert (1966, 1969 b) have presented computing algorithms for converting a brood class of optimal control problems (including minimum time, and fixed-time minimum fuel, energy and effort problems) to a sequence of optimal regulator problems, using a one dimensional search of the cost variable. These Barr and Gilbert algorithms, which use quadratic programming algorithms by the same authors (1969 a) to solve the resulting optimal regulator problems, are restricted to dynamic equations linear in state by virtue of using the convexity and compactness (Neustadt 1963) and contact function (Gilbert 1966) of the reachable set

This paper extends the above approach to a class of terminal cost optimal control problems similar to those considered by Barr and Gilbert (including quite general control constraints, but only allowing initial and final state constraints), having differential equations non-linear instate and control (where the convexity-compactness results do not hold), by converting each such problem to a sequence of optimal regulator problems, with non-linear differential equations. These, in turn, are solved by one of the author's earlier algorithms (Katz 1974) that makes use of the above convexity, compactness, and contact function results by repeatedly linearizing the regulator problems. The approach of this paper differs from that of Halkin (1964 b), in that Halkin directly linearizes the original problem (e.g. converting a non-linear minimum fuel problem to a linear minimum fuel problem) and then solves the linearized version by a doubly iterative procedure

The computing algorithm presented here is based on the definition of an appropriate approximate solution of the terminal cost problem. A local-minimum convergence proof is given, which is weak in the sense that it assumes convergence of the substep algorithm (Katz 1974) for non-linear optimal regulator problems, whose convergence has not been proved. A subsequent paper (Katz and Wachtor, to appear) shows good convergence of the (overall) terminal cost problem algorithm in examples having singular arcs, with no prior knowledge of the solution or its singular nature, other than an initial upper bound on the cost.     

Convergence analysis of a computational method for optimal control of non-linear differential-algebraic systems

The convergence analysis of a computational method for optimal control problems of non-linear differential-algebraic systems is considered. The class of admissible controls is taken to be the class of piecewise smooth functions. A control parametrizution technique is used to approximate the optimal control problem into a sequence of optimal parameter selection problems. The solution of each of these approximate problems gives rise to a suboptimal solution to the original optimal control problem in an obvious way. The gradients of the cost functional with respect to parameters are derived. Furthermore, the error bounds between the suboptimal costs and the true optimal cost are derived.     

Application of dynamic programming to high-dimensional non-linear optimal control problems

In using dynamic programming, by taking only accessible states for the x-grid and using an iterative procedure employing region contraction, only a small number of grid points are required at each iteration to yield very good accuracy even if the dimension of the system is high. The effect of the number of grid points and the choice of the contraction factor are analysed by considering a non-linear system consisting of eight ordinary differential equations and four control variables.

No difficulties were encountered in convergence to the optimal solution in no more than 20 iterations. The proposed procedure overcomes the curse of dimensionality that has discouraged the use of dynamic programming in the past to solve high-dimensional non-linear optimal control problems, and provides an attractive means of solving optimal control problems in general     

Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms

An important application of reinforcement learning (RL) is to finite-state control problems and one of the most difficult problems in learning for control is balancing the exploration/exploitation tradeoff. Existing theoretical results for RL give very little guidance on reasonable ways to perform exploration. In this paper, we examine the convergence of single-step on-policy RL algorithms for control. On-policy algorithms cannot separate exploration from learning and therefore must confront the exploration problem directly. We prove convergence results for several related on-policy algorithms with both decaying exploration and persistent exploration. We also provide examples of exploration strategies that can be followed during learning that result in convergence to both optimal values and optimal policies.     

Turnpike properties of a class of aquifer control problems

In this paper we are concerned with the long-run behavior of solutions to infinite horizon optimal control systems, where the state equation is given by a non-linear parabolic equation. The goal is to establish convergence to steady state of solutions of the optimal control problems under consideration. Our motivation to consider this problem is the study of agricultural economics models concerned with the optimal management of groundwater. The main object of the study towards achieving this goal is the continuity of solutions of the state equation with respect to the initial condition and the non-homogeneous term. We establish this continuity by employing well-known norm estimates for linear parabolic equations.     

Repetitive control of Hamiltonian systems based on variational symmetry

This paper is concerned with repetitive control of Hamiltonian systems, which is based on iterative learning control utilizing the variational symmetry of those systems. Variational symmetry allows us to obtain an algorithm to solve a certain class of optimal control problems in a repetitive control framework. Therefore, the proposed method can deal with not only trajectory tracking control problems but also optimal trajectory generation problems, never before considered in a repetitive control framework. A convergence analysis of this algorithm is also discussed. Furthermore, some numerical simulations demonstrate the effectiveness of the proposed method.     

Control Liapunov function design of neural networks that solve convex optimization and variational inequality problems

This paper presents two neural networks to find the optimal point in convex optimization problems and variational inequality problems, respectively. The domain of the functions that define the problems is a convex set, which is determined by convex inequality constraints and affine equality constraints. The neural networks are based on gradient descent and exact penalization and the convergence analysis is based on a control Liapunov function analysis, since the dynamical system corresponding to each neural network may be viewed as a so-called variable structure closed loop control system.     

Optimal periodic control problem: a modified trigonometric approximation and quasi-stationarity conditions

The constrained optimal periodic control problem for a system described by differential equations and endowed with inertial controllers is considered, A sequence of discretized problems using trigonometric polynomials is proposed to approximate the problem. Instantaneous constraints for the state and control are handled by a new and more precise approach that imposes only a small number of non-linear but easily computable constraints. The convergence conditions for a sequence of optimal solutions of discretized problems are derived. The inclusion in the approximating scheme of various quasi-stationarity conditions for the control and state variables is analysed. Extension of a new approximating approach for inertialess and smooth problems is also discussed.     

Penalty-Methode bei der numerischen Behandlung von quasilinearen elliptischen Differentialgleichungen vierter und höherer Ordnung

In the following paper we treat the numerical solution of quasilinear elliptic differential equations of fourth and higher order which are Euler-equations of certain variational problems We reduce the differential equation to a system of equations of the second order and solve this system by the method of finite differences. Existence and uniqueness of a minimal solution of the discrete problem and convergence to the solution of the variational problem under the assumptions of consistency and stability are established as the mesh size and the Penalty-parameter tend to zero.     

Solutions of nonlinear control and estimation problems in reproducing kernel Hilbert spaces: Existence and numerical determination

Many nonlinear optimal control and estimation problems can be formulated as Tikhonov variational problems in an infinite dimensional reproducing kernel Hilbert space. This paper shows that any closed ball contained in such spaces is compact in the sup-norm topology. This result is exploited to obtain conditions which guarantee existence of solutions for the aforementioned problems as well as numerical algorithms whose convergence is guaranteed in the space of continuous functions.     

A Level Set Model for Image Classification

We present a supervised classification model based on a variational approach. This model is devoted to find an optimal partition composed of homogeneous classes with regular interfaces. The originality of the proposed approach concerns the definition of a partition by the use of level sets. Each set of regions and boundaries associated to a class is defined by a unique level set function. We use as many level sets as different classes and all these level sets are moving together thanks to forces which interact in order to get an optimal partition. We show how these forces can be defined through the minimization of a unique fonctional. The coupled Partial Differential Equations (PDE) related to the minimization of the functional are considered through a dynamical scheme. Given an initial interface set (zero level set), the different terms of the PDE's are governing the motion of interfaces such that, at convergence, we get an optimal partition as defined above. Each interface is guided by internal forces (regularity of the interface), and external ones (data term, no vacuum, no regions overlapping). Several experiments were conducted on both synthetic and real images.     

Finite-dimensional regularizers for optimal control problems on solutions of ill-posed variational inequalities

A method is proposed for construction of finite-dimensional regularizers for optimal control on solutions of ill-posed variational inequalities with pseudomonotone operators. It is assumed that all the input data of the problems, including the control sets, are known approximately.Translated from Kibernetika i Sistemnyi Analiz, No. 4, pp. 84–88, July–August, 1991.     

基于伪谱法的滑翔轨迹多级迭代优化策略

伪谱法可实时求解具有高度非线性动态特性的飞行器最优轨迹;以X-51A相似飞行器模型为研究对象,采用增量法与查表插值建立纵向气动力模型,伪谱法与序列二次规划算法求解滑翔轨迹最优控制问题;提出使用多级迭代优化策略,为序列二次规划算法求解伪谱法参数化得到的大规模非线性规划问题提供初值,弥补序列二次规划算法在求解大规模非线性规划问题过程中,出现的初值敏感、收敛速度减慢等问题。通过与传统方法求解出的状态量与控制量仿真飞行状态进行对比,证明了多级迭代优化策略的有效性和高效性,该策略在实际工程应用中取得了良好效果。     

Measure-Valued Variational Models with Applications to Diffusion-Weighted Imaging

We develop a general mathematical framework for variational problems where the unknown function takes values in the space of probability measures on some metric space. We study weak and strong topologies and define a total variation seminorm for functions taking values in a Banach space. The seminorm penalizes jumps and is rotationally invariant under certain conditions. We prove existence of a minimizer for a class of variational problems based on this formulation of total variation and provide an example where uniqueness fails to hold. Employing the Kantorovich–Rubinstein transport norm from the theory of optimal transport, we propose a variational approach for the restoration of orientation distribution function-valued images, as commonly used in diffusion MRI. We demonstrate that the approach is numerically feasible on several data sets.     

A Mixed Finite Element Scheme for Optimal Control Problems with Pointwise State Constraints

In this paper, we propose a mixed variational scheme for optimal control problems with point-wise state constraints, the main idea is to reformulate the optimal control problems to a constrained minimization problem involving only the state, which is characterized by a fourth order variational inequality. Then mixed form based on this fourth order variational inequality is formulated and a direct numerical algorithm is proposed without the optimality conditions of underlying optimal control problems. The a priori and a posteriori error estimates are proved for the mixed finite element scheme. Numerical experiments confirm the efficiency of the new strategy.     

A Neural Network Approach for Solving Optimal Control Problems with Inequality Constraints and Some Applications

In this paper, a class of nonlinear optimal control problems with inequality constraints is considered. Based on Karush–Kuhn–Tucker optimality conditions of nonlinear optimization problems and by constructing an error function, we define an unconstrained minimization problem. In the minimization problem, we use trial solutions for the state, Lagrange multipliers, and control functions where these trial solutions are constructed by using two-layered perceptron. We then minimize the error function using a dynamic optimization method where weights and biases associated with all neurons are unknown. The stability and convergence analysis of the dynamic optimization scheme is also studied. Substituting the optimal values of the weights and biases in the trial solutions, we obtain the optimal solution of the original problem. Several examples are given to show the efficiency of the method. We also provide two applicable examples in robotic engineering.