Optimal control problem via neural networks

This paper attempts to propose a new method based on capabilities of artificial neural networks, in function approximation, to attain the solution of optimal control problems. To do so, we try to approximate the solution of Hamiltonian conditions based on the Pontryagin minimum principle (PMP). For this purpose, we introduce an error function that contains all PMP conditions. In the proposed error function, we used trial solutions for the trajectory function, control function and the Lagrange multipliers. These trial solutions are constructed by using neurons. Then, we minimize the error function that contains just the weights of the trial solutions. Substituting the optimal values of the weights in the trial solutions, we obtain the optimal trajectory function, optimal control function and the optimal Lagrange multipliers.     

Learning shape from shading by a multilayer network

The multilayer feedforward network has often been used for learning a nonlinear mapping based on a set of examples of the input-output data. In this paper, we present a novel use of the network, in which the example data are not explicitly given. We consider the problem of shape from shading in computer vision, where the input (image coordinates) and the output (surface depth) satisfy only a known differential equation. We use the feedforward network as a parametric representation of the object surface and reformulate the shape from shading problem as the minimization of an error function over the network weights. The stochastic gradient and conjugate gradient methods are used for the minimization. Boundary conditions for either surface depth or surface normal (or both) can be imposed by adjusting the same network at different levels. It is further shown that the light source direction can be estimated, based on an initial guess, by integrating the source estimation with the surface estimation. Extensions of the method to a wider class of problems are discussed. The efficiency of the method is verified by examples of both synthetic and real images.     

Interior point techniques for optimal control of variational inequalities

This paper is devoted to a new application of an interior point algorithm to solve optimal control problems of variational inequalities. We propose a Lagrangian technique to obtain a necessary optimality system. After the discretization of the optimality system we prove its equivalence to Karush-Kuhn-Tucker conditions of a nonlinear regular minimization problem. This problem can be efficiently solved by using a modification of Herskovits' interior point algorithm for nonlinear optimization. We describe the numerical scheme for solving this problem and give some numerical examples of test problems in 1-D and 2-D.     

Necessary and sufficient conditions for optimality inlinear-quadratic problems of optimal control

Linear-quadratic Bolza problems of optimal control with variable end points are considered. Under the strengthened Legendre condition, necessary and sufficient optimality conditions are established, and it is shown that the linear-quadratic Bolza problem of optimal control can be reduced to a quadratic minimization problem in a finite-dimensional space. Simple simulations where solutions of a nonlinear problem can be recovered from solutions of the accessory linear-quadratic problem are indicated. Conjectures regarding sufficient conditions for optimality in nonlinear Bolza problems are included     

Approximating the Solution of Optimal Control Problems by Fuzzy Systems

In this paper, the ability of fuzzy systems is used to estimate the solution of crisp optimal control problems. To solve an optimal control problem, first the well-known Euler–Lagrange conditions are obtained and then, the solution of these conditions is approximated by defining a trial solution based on fuzzy systems. The parameters of fuzzy systems are adjusted by an optimization algorithm. Numerical examples and comparisons with exact solutions reveal the capability and accuracy of proposed method.     

Maximizing deviation method for neutrosophic multiple attribute decision making with incomplete weight information

This paper develops a method for solving the multiple attribute decision-making problems with the single-valued neutrosophic information or interval neutrosophic information. We first propose two discrimination functions referred to as score function and accuracy function for ranking the neutrosophic numbers. An optimization model to determine the attribute weights that are partly known is established based on the maximizing deviation method. For the special situations where the information about attribute weights is completely unknown, we propose another optimization model. A practical and useful formula which can be used to determine the attribute weights is obtained by solving a proposed nonlinear optimization problem. To aggregate the neutrosophic information corresponding to each alternative, we utilize the neutrosophic weighted averaging operators which are the single-valued neutrosophic weighted averaging operator and the interval neutrosophic weighted averaging operator. Thus, we can determine the order of alternatives and choose the most desirable one(s) based on the score function and accuracy function. Finally, some illustrative examples are presented to verify the proposed approach and to present its effectiveness and practicality.     

A compliance based design problem of structures under multiple load cases

There are two popular methods concerning the optimal design of structures. The first is the minimization of the volume of the structure under stress constraints. The second is the minimization of the compliance for a given volume. For multiple load cases an arising issue is which energy quantity should be the objective function. Regarding the sizing optimization of trusses, Rozvany proved that the solution of the established compliance based problems leads to results which are awkward and not equivalent to the solutions of minimization of the volume under stress constraints, unlike under single loading (the layouts would be the same if in the compliance problem the volume is set equal to the result of the first problem). In this paper, we introduce the “envelope strain energy” problem where we minimize the volume integral of the worst case strain energy of each point of the structure. We also prove that in the case of sizing optimization of statically non-indeterminate (the term non-indeterminate includes both statically determinate trusses and mechanisms) trusses, this compliance method gives the same optimal design as the stress based design method.     

Synthesis of fault-tolerant feedforward neural networks usingminimax optimization

In this paper we examine a technique by which fault tolerance can be embedded into a feedforward network leading to a network tolerant to the loss of a node and its associated weights. The fault tolerance problem for a feedforward network is formulated as a constrained minimax optimization problem. Two different methods are used to solve it. In the first method, the constrained minimax optimization problem is converted to a sequence of unconstrained least-squares optimization problems, whose solutions converge to the solution of the original minimax problem. An efficient gradient-based minimization technique, specially tailored for nonlinear least-squares optimization, is then applied to perform the unconstrained minimization at each step of the sequence. Several modifications are made to the basic algorithm to improve its speed of convergence. In the second method a different approach is used to convert the problem to a single unconstrained minimization problem whose solution very nearly equals that of the original minimax problem. Networks synthesized using these methods, though not always fault tolerant, exhibit an acceptable degree of partial fault tolerance.     

Improved hybrid method as a robust tool for reliability-based design optimization

In the engineering problems, the randomness and the uncertainties of the distribution of the structural parameters are a crucial problem. In the case of reliability-based design optimization (RBDO), it is the objective to play a dominant role in the structural optimization problem introducing the reliability concept. The RBDO problem is often formulated as a minimization of the initial structural cost under constraints imposed on the values of elemental reliability indices corresponding to various limit states. The classical RBDO leads to high computing time and weak convergence, but a Hybrid Method (HM) has been proposed to overcome these two drawbacks. As the hybrid method successfully reduces the computing time, we can increase the number of variables by introducing the standard deviations as optimization variables to minimize the error values in the probabilistic model. The efficiency of the hybrid method has been demonstrated on static and dynamic cases with extension to the variability of the probabilistic model. In this paper, we propose a modification on the formulation of the hybrid method to improve the optimal solutions. The proposed method is called, Improved Hybrid Method (IHM). The main benefit of this method is to improve the structure performance by much more minimizing the objective function than the hybrid method. It is also shown to demonstrate the optimality conditions. The improved hybrid method is next applied to two numerical examples, with consideration of the standard deviations as optimization variables (for linear and nonlinear distributions). When integrating the improved hybrid method within the probabilistic model variability, we minimize the objective function more and more.     

Selecting a restoration technique to minimize OCR error

This paper introduces a learning problem related to the task of converting printed documents to ASCII text files. The goal of the learning procedure is to produce a function that maps documents to restoration techniques in such a way that on average the restored documents have minimum optical character recognition error. We derive a general form for the optimal function and use it to motivate the development of a nonparametric method based on nearest neighbors. We also develop a direct method of solution based on empirical error minimization for which we prove a finite sample bound on estimation error that is independent of distribution. We show that this empirical error minimization problem is an extension of the empirical optimization problem for traditional M-class classification with general loss function and prove computational hardness for this problem. We then derive a simple iterative algorithm called generalized multiclass ratchet (GMR) and prove that it produces an optimal function asymptotically (with probability 1). To obtain the GMR algorithm we introduce a new data map that extends Kesler's construction for the multiclass problem and then apply an algorithm called Ratchet to this mapped data, where Ratchet is a modification of the Pocket algorithm . Finally, we apply these methods to a collection of documents and report on the experimental results.     

Two-loop reinforcement learning algorithm for finite-horizon optimal control of continuous-time affine nonlinear systems

This article proposes three novel time-varying policy iteration algorithms for finite-horizon optimal control problem of continuous-time affine nonlinear systems. We first propose a model-based time-varying policy iteration algorithm. The method considers time-varying solutions to the Hamiltonian–Jacobi–Bellman equation for finite-horizon optimal control. Based on this algorithm, value function approximation is applied to the Bellman equation by establishing neural networks with time-varying weights. A novel update law for time-varying weights is put forward based on the idea of iterative learning control, which obtains optimal solutions more efficiently compared to previous works. Considering that system models may be unknown in real applications, we propose a partially model-free time-varying policy iteration algorithm that applies integral reinforcement learning to acquiring the time-varying value function. Moreover, analysis of convergence, stability, and optimality is provided for every algorithm. Finally, simulations for different cases are given to verify the convenience and effectiveness of the proposed algorithms.     

桥式吊车系统的伪谱最优控制设计

对于桥式吊车系统的最优控制问题,根据实际的工况要求,性能指标有时不一定是标准的二次形式.同时,在实际的控制问题中,状态和控制输入往往会受到一些边界条件和路径过程中的约束.针对这一问题,本文应用Chebyshev伪谱优化算法来处理,它可以处理状态和控制约束的非线性最优化问题以及一个非标准的目标函数.首先对桥式吊车系统模型进行一系列的坐标变换,将其转变为上三角系统形式的误差模型.然后将桥式吊车最优控制问题转化成具有一系列代数约束的参数优化问题,即非线性规划问题.通过求解离散化后的参数优化问题,得到桥式吊车的最优控制律.本文还给出了Chebyshev伪谱最优解的可行性和一致性分析.最后,在仿真研究中验证该控制器的有效性.     

Optimal power distribution control for a network of fuel cell stacks

In power networks, where multiple fuel cell stacks are employed in a series-parallel configuration to deliver the required power, optimal sharing of the power demand between different stacks is an important problem. This is because the total current collectively produced by all the stacks is directly proportional to the fuel utilization, through stoichiometry. As a result, one would like to produce the required power while minimizing the total current produced. In this paper, an optimization formulation is proposed for this power distribution control problem. An algorithm that identifies the globally optimal solution for this problem is developed. Through an analysis of the KKT conditions, the solution to the optimization problem is decomposed into off-line and on-line computations. The on-line computations reduce to solving a nonlinear equation. For an application with a specific V–I function derived from data, we show that analytical solutions exist for on-line computations. We also discuss the wider applicability of the proposed approach for similar problems in other domains.     

Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning

In this paper, a new formulation for the optimal tracking control problem (OTCP) of continuous-time nonlinear systems is presented. This formulation extends the integral reinforcement learning (IRL) technique, a method for solving optimal regulation problems, to learn the solution to the OTCP. Unlike existing solutions to the OTCP, the proposed method does not need to have or to identify knowledge of the system drift dynamics, and it also takes into account the input constraints a priori. An augmented system composed of the error system dynamics and the command generator dynamics is used to introduce a new nonquadratic discounted performance function for the OTCP. This encodes the input constrains into the optimization problem. A tracking Hamilton–Jacobi–Bellman (HJB) equation associated with this nonquadratic performance function is derived which gives the optimal control solution. An online IRL algorithm is presented to learn the solution to the tracking HJB equation without knowing the system drift dynamics. Convergence to a near-optimal control solution and stability of the whole system are shown under a persistence of excitation condition. Simulation examples are provided to show the effectiveness of the proposed method.     

Impulsive Synchronization of Stochastic Neural Networks via Controlling Partial States

In this paper the perceptron neural networks are applied to approximate the solution of fractional optimal control problems. The necessary (and also sufficient in most cases) optimality conditions are stated in a form of fractional two-point boundary value problem. Then this problem is converted to a Volterra integral equation. By using perceptron neural network’s ability in approximating a nonlinear function, first we propose approximating functions to estimate control, state and co-state functions which they satisfy the initial or boundary conditions. The approximating functions contain neural network with unknown weights. Using an optimization approach, the weights are adjusted such that the approximating functions satisfy the optimality conditions of fractional optimal control problem. Numerical results illustrate the advantages of the method.     

A generalized Suzuki–Trotter type method in optimal control of coupled Schrödinger equations

A generalized Suzuki–Trotter (GST) method for the solution of an optimal control problem for quantum molecular systems is presented in this work. The control of such systems gives rise to a minimization problem with constraints given by a system of coupled Schrödinger equations. The computational bottleneck of the corresponding minimization methods is the solution of time-dependent Schrödinger equations. To solve the Schrödinger equations we use the GST framework to obtain an explicit polynomial approximation of the matrix exponential function. The GST method almost exclusively uses the action of the Hamiltonian and is therefore efficient and easy to implement for a variety of quantum systems. Following a first discretize, then optimize approach we derive the correct discrete representation of the gradient and the Hessian. The derivatives can naturally be expressed in the GST framework and can therefore be efficiently computed. By recomputing the solutions of the Schrödinger equations instead of saving the whole time evolution, we are able to significantly reduce the memory requirements of the method at the cost of additional computations. This makes first and second order optimization methods viable for large scale problems. In numerical experiments we compare the performance of different first and second order optimization methods using the GST method. We observe fast local convergence of second order methods.     

Unconditionally Optimal Error Estimates of a Linearized Galerkin Method for Nonlinear Time Fractional Reaction–Subdiffusion Equations

This paper is concerned with unconditionally optimal error estimates of linearized Galerkin finite element methods to numerically solve some multi-dimensional fractional reaction–subdiffusion equations, while the classical analysis for numerical approximation of multi-dimensional nonlinear parabolic problems usually require a restriction on the time-step, which is dependent on the spatial grid size. To obtain the unconditionally optimal error estimates, the key point is to obtain the boundedness of numerical solutions in the \(L^\infty \)-norm. For this, we introduce a time-discrete elliptic equation, construct an energy function for the nonlocal problem, and handle the error summation properly. Compared with integer-order nonlinear problems, the nonlocal convolution in the time fractional derivative causes much difficulties in developing and analyzing numerical schemes. Numerical examples are given to validate our theoretical results.     

Solving Fixed Final Time Fractional Optimal Control Problems Using the Parametric Optimization Method

In this paper, the parametric optimization method is used to find optimal control laws for fractional systems. The proposed approach is based on the use for the fractional variational iteration method to convert the original optimal control problem into a nonlinear optimization one. The control variable is parameterized by unknown parameters to be determined, then its expression is substituted into the system state‐space model. The resulting fractional ordinary differential equations are solved by the fractional variational iteration method, which provides an approximate analytical expression of the closed‐form solution of the state equations. This solution is a function of time and the unknown parameters of the control law. By substituting this solution into the performance index, the original fractional optimal control problem reduces to a nonlinear optimization problem where the unknown parameters, introduced in the parameterization procedure, are the optimization variables. To solve the nonlinear optimization problem and find the optimal values of the control parameters, the Alienor global optimization method is used to achieve the global optimal values of the control law parameters. The proposed approach is illustrated by two application examples taken from the literature.     

Convexity and convex approximations of discrete-time stochastic control problems with constraints

We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost subject to probabilistic constraints. We study the convexity of a finite-horizon optimization problem in the case where the control policies are affine functions of the disturbance input. We propose an expectation-based method for the convex approximation of probabilistic constraints with polytopic constraint function, and a Linear Matrix Inequality (LMI) method for the convex approximation of probabilistic constraints with ellipsoidal constraint function. Finally, we introduce a class of convex expectation-type constraints that provide tractable approximations of the so-called integrated chance constraints. Performance of these methods and of existing convex approximation methods for probabilistic constraints is compared on a numerical example.     

Guaranteed cost control of affine nonlinear systems via partition of unity method

We consider the problem of guaranteed cost control (GCC) of affine nonlinear systems in this paper. Firstly, the general affine nonlinear system with the origin being its equilibrium point is represented as a linear-like structure with state-dependent coefficient matrices. Secondly, partition of unity method is used to approximate the coefficient matrices, as a result of which the original affine nonlinear system is equivalently converted into a linear-like system with modeling error. A GCC law is then synthesized based on the equivalent model in the presence of modeling error under certain error condition. The control law ensures that the system under control is asymptotically stable as well as that a given cost function is upper-bounded. A suboptimal GCC law can be obtained via solving an optimization problem in terms of linear matrix inequality (LMI), in stead of state-dependent Riccati equation (SDRE) or Hamilton–Jacobi equations that are usually required in solving nonlinear optimal control problems. Finally, a numerical example is provided to illustrate the validity of the proposed method.