Control of singularly perturbed hybrid stochastic systems

We study a class of optimal stochastic control problems involving two different time scales. The fast mode of the system is represented by deterministic state equations whereas the slow mode of the system corresponds to a jump disturbance process. Under a fundamental “ergodicity” property for a class of “infinitesimal control systems” associated with the fast mode, we show that there exists a limit problem which provides a good approximation to the optimal control of the perturbed system. Both the finite- and infinite-discounted horizon cases are considered. We show how an approximate optimal control law can be constructed from the solution of the limit control problem. In the particular case where the infinitesimal control systems possess the so-called turnpike property, i.e., characterized by the existence of global attractors, the limit control problem can be given an interpretation related to a decomposition approach     

Optimal control of pull manufacturing systems

We consider the problem of optimal control of pull manufacturing systems. We study a fluid model of a flow shop, with buffer holding costs nondecreasing along the route. The system is subject to a constant exogenous demand, thus incurring additional shortfall/inventory costs. The objective is to determine the optimal control for the production rate at each machine in the system. We exhibit a decomposition of the flow shop into “sections” of contiguous machines, where, in each section, the head machine is the bottleneck for the downstream system. We exhibit the form of an optimal control and show that it is characterized by a set of “deferral times”, one for each head machine. Machines which are upstream of a head machine simply adopt a “just-in-time” production policy. The head machines initially stay idle for a period equal to their deferral time and thereafter produce as fast as possible, until the initial shortfall is eliminated. The optimal values of these deferral times are simply obtained by solving a set of quadratic programming problems. We also exhibit special cases of re-entrant lines, for which the optimal control is similarly computable     

Domain decomposition and model reduction for the numerical solution of PDE constrained optimization problems with localized optimization variables

We introduce a technique for the dimension reduction of a class of PDE constrained optimization problems governed by linear time dependent advection diffusion equations for which the optimization variables are related to spatially localized quantities. Our approach uses domain decomposition applied to the optimality system to isolate the subsystem that explicitly depends on the optimization variables from the remaining linear optimality subsystem. We apply balanced truncation model reduction to the linear optimality subsystem. The resulting coupled reduced optimality system can be interpreted as the optimality system of a reduced optimization problem. We derive estimates for the error between the solution of the original optimization problem and the solution of the reduced problem. The approach is demonstrated numerically on an optimal control problem and on a shape optimization problem.     

A heuristic approach to solving a class of bilinear matrix inequality problems

Many canonical and modern control problems can be recast into the problem of a group of matrix inequalities. Some of them are in the form of linear matrix inequalities (LMIs), which can be solved very efficiently by the powerful LMI toolbox in Matlab, but some others are in the form of bilinear matrix inequalities. The characteristic of this latter class of problems is that when the so called “communicating variables” are fixed, the overall problem will be reduced to the problem in LMIs. Thus, how to find the communicating variables is the key to solve the whole problem. In this paper, an optimal estimate for the communicating variables is presented. We will illustrate our method by completely solving the problems of overshoot bound control and reachable set analysis for uncertain systems. Numerical examples are provided to show the effectiveness of the proposed method.     

Constrained Optimal Hybrid Control of a Flow Shop System

We consider an optimal control problem for the hybrid model of a deterministic flow shop system, in which the jobs are processed in the order they arrive at the system. The problem is decomposed into a higher-level discrete-event system control problem of determining the optimal service times, and a set of lower-level classical control problems of determining the optimal control inputs for given service times. We focus on the higher-level problem which is nonconvex and nondifferentiable. The arrival times are known and the decision variables are the service times that are controllable within constraints. We present an equivalent convex optimization problem with linear constraints. Under some cost assumptions, we show that no waiting is observed on the optimal sample path. This property allows us to simplify the convex optimization problem by eliminating variables and constraints. We also prove, under an additional strict convexity assumption, the uniqueness of the optimal solution and propose two algorithms to decompose the simplified convex optimization problem into a set of smaller convex optimization problems. The effects of the simplification and the decomposition on the solution times are shown on an example problem.     

A polynomial approach for optimal control of switched nonlinear systems

Optimal control problems for switched nonlinear systems are investigated. We propose an alternative approach for solving the optimal control problem for a nonlinear switched system based on the theory of moments. The essence of this method is the transformation of a nonlinear, nonconvex optimal control problem, that is, the switched system, into an equivalent optimal control problem with linear and convex structure, which allows us to obtain an equivalent convex formulation more appropriate to be solved by high‐performance numerical computing. Consequently, we propose to convexify the control variables by means of the method of moments obtaining semidefinite programs. Copyright © 2013 John Wiley & Sons, Ltd.     

Consistency issues for numerical methods for variance control, withapplications to optimization in finance

The paper is concerned with numerical algorithms for the optimal control of diffusion-type processes when the noise variance also depends on the control. This problem is of increasing importance in applications, particularly in financial mathematics. We discuss the construction of numerical algorithms guaranteed to converge to the true minimum as the discretization level decreases and with acceptable numerical properties. The algorithms are based on the popular Markov chain approximation method. The basic criterion the algorithms must satisfy is a weak “local consistency” condition, which is essential for convergence to the true optimal cost function. This condition is often hard to satisfy by simple algorithms (with essentially only local transitions) when the variance is also controlled. Numerical “noise” can be introduced by the more convenient approximations. This question of “numerical noise” (also called “numerical viscosity”) is dealt with in detail, and methods for eliminating or greatly reducing it are discussed     

Generalized l2 synthesis

Introduces a general framework for control design which can be used to solve several open problems in robust and optimal control. These include systems subject to independently norm-bounded disturbances, robust stability for systems subject to “element by element” bounded structured uncertainty, certain classes of robust performance problems, and systems subject to deterministic white noise disturbances. This is achieved by combining concepts such as H-infinity optimization, linear matrix inequalities, and integral quadratic constraints. Necessary and sufficient conditions for the general problem to have a solution are in terms of a computationally attractive finite-dimensional linear matrix inequality     

Neural approximations for multistage optimal control of nonlinearstochastic systems

Two main approximations are used to solve a nonlinear-quadratic-Gaussian (LQG) optimal control problem: the control law is assigned a given structure in which a finite number of parameters have to be determined to minimize the cost function (the chosen structure is that of a multilayer feedforward neural network); and the control law is given a “limited memory”. The errors resulting front both assumptions are discussed. Simulation results show that the proposed method may constitute a simple and effective tool for solving, to a sufficient degree of accuracy, optimal control problems traditionally regarded as difficult ones     

On the open-loop solution of linear stochastic optimal control problems

We consider open-loop solutions of linear stochastic optimal control problems with constraints on control variables and probabilistic constraints on state variables. It is shown that this problem reduces to an equivalent linear deterministic optimal control problem with similar constraints and with a new criterion to minimize. Concavity or convexity is preserved. Hence, the machinery available for solving deterministic optimal control problems can be used to get an open-loop solution of the stochastic problem. The convex case is investigated and a bound on the difference between closed-loop and open-loop optimal costs is given.     

State-space constrained optimal control problems with control variables appearing linearly

This paper deals with the state-space constrained optimal control problems with control variables appearing linearly by the concept of decomposition. To solve this continuous optimal control problem, we first discretize the time and replace the system of differential equations by difference equations. For this resulting discrete optimal control problem, fixing the value of state variables reduces the given problem to a finite number of independent linear programming problems which are parameterized by the value of state variables. From this point of view, after para. meterizing by the value of state variables, we outer-linearize the resulting itifimal valuo functions in the minimond and apply the relaxation strategy to the new constraints arising as a consequence of outer-linearization. An algorithm is proposed which requires baek-and-forth iteration between a master problem and a finite number of linear programming subproblems. Finite convergence of this algorithm follows directly from the finite number of constraints of the master problem.     

Remarks on the paper “Optimal control of a class of linear multivariable systems with integral quadratic energy constraint ”

An optimal periodic control problem for a system described by differential equations is considered. Control units are assumed to generate control actions with the square-integrable derivative. The above problem is approximated by a sequence of discretized problems containing trigonometric polynomials, which approximate the state and control variables, and the functions in the criterion and differential equations. The conditions for a sequence of optimal solutions to discretized problems, which are to be a generalized minimizing sequence for the basic problem, are given. Extensions to more general problem formulations are presented. The possibility of application is illustrated by the example of an optimal periodic control problem for a chemical reactor.     

Certified Reduced Basis Methods for Parametrized Elliptic Optimal Control Problems with Distributed Controls

In this paper, we consider the efficient and reliable solution of distributed optimal control problems governed by parametrized elliptic partial differential equations. The reduced basis method is used as a low-dimensional surrogate model to solve the optimal control problem. To this end, we introduce reduced basis spaces not only for the state and adjoint variable but also for the distributed control variable. We also propose two different error estimation procedures that provide rigorous bounds for the error in the optimal control and the associated cost functional. The reduced basis optimal control problem and associated a posteriori error bounds can be efficiently evaluated in an offline–online computational procedure, thus making our approach relevant in the many-query or real-time context. We compare our bounds with a previously proposed bound based on the Banach–Ne?as–Babu?ka theory and present numerical results for two model problems: a Graetz flow problem and a heat transfer problem. Finally, we also apply and test the performance of our newly proposed bound on a hyperthermia treatment planning problem.     

Optimal feedback controls in deterministic two-machine flowshopswith finite buffers

This paper deals with an optimal control problem of deterministic two-machine flowshops. Since the sizes of both internal and external buffers are practically finite, the problem is one with state constraints. The Hamilton-Jacobi-Bellman (HJB) equations of the problem involve complicated boundary conditions due to the presence of the state constraints, and as a consequence the usual “verification theorem” may not work for the problem. To overcome this difficulty, it is shown that any function satisfying the HJB equations in the interior of the state constraint domain must be majorized by the value function. The main techniques employed are the “constraint domain approximation” approach and the “weak-Lipschitz” property of the value functions developed in preceding papers. Based on this, an explicit optimal feedback control for the problem is obtained     

High-fidelity real-time simulation on deployed platforms

We present a certified reduced basis method for high-fidelity real-time solution of parametrized partial differential equations on deployed platforms. Applications include in situ parameter estimation, adaptive design and control, interactive synthesis and visualization, and individuated product specification. We emphasize a new hierarchical architecture particularly well suited to the reduced basis computational paradigm: the expensive Offline stage is conducted pre-deployment on a parallel supercomputer (in our examples, the TeraGrid machine Ranger); the inexpensive Online stage is conducted “in the field” on ubiquitous thin/inexpensive platforms such as laptops, tablets, smartphones (in our examples, the Nexus One Android-based phone), or embedded chips. We illustrate our approach with three examples: a two-dimensional Helmholtz acoustics “horn” problem; a three-dimensional transient heat conduction “Swiss Cheese” problem; and a three-dimensional unsteady incompressible Navier–Stokes low-Reynolds-number “eddy-promoter” problem.     

A computational method for solving time-delay optimal control problems with free terminal time

This paper considers a class of optimal control problems for general nonlinear time-delay systems with free terminal time. We first show that for this class of problems, the well-known time-scaling transformation for mapping the free time horizon into a fixed time interval yields a new time-delay system in which the time delays are variable. Then, we introduce a control parameterization scheme to approximate the control variables in the new system by piecewise-constant functions. This yields an approximate finite-dimensional optimization problem with three types of decision variables: the control heights, the control switching times, and the terminal time in the original system (which influences the variable time delays in the new system). We develop a gradient-based optimization approach for solving this approximate problem. Simulation results are also provided to demonstrate the effectiveness of the proposed approach.     

Optimal almost-periodic control problem: a trigonometric approximation approach

The problem of finding an almost-periodic control that is optimal with respect to a certain time-averaged criterion for the dynamic system operated over a long period of time is considered. The existence of the optimal solution, spectral properties of which satisfy certain regularity conditions, is hypothesized. The problem is approximated by a sequence of finite-dimensional optimization problems containing trigonometric sums for the approximation of the state and control variables, and using a Fejér-Riesz type representation for a positive trigonometric sum to handle the instantaneous constraints for these variables. Sufficient conditions for the sequence of approximate optimal solutions of the discretized problems to be an approximately minimizing sequence for the basic problem are given. The constructive character of the proposed approach and its potential applications are pointed out both for dynamic systems affected by irregularly pulsating disturbances and for stationary systems, the non-linear dynamics of which can be exploited by a non-stationary control to improve the averaged system performance.     

An efficient algorithm for solving optimal control problems with linear terminal constraints

The optimization of nonlinear systems subject to linear terminal state variable constraints is considered. A technique for solving this class of problems is proposed that involves a piecewise polynomial parameterization of the system variables. The optimal control problem is thereby reduced to a linearly constrained parameter optimization problem which can be solved efficiently using the quadratically convergent Gold-farb-Lapidus algorithm. Illustrative numerical examples are presented.     

A quasilinearization algorithm and its application to a manipulator problem

A quasilinearization algorithm is proposed for the computation of optimal control of a class of constrained problem. The constraints are inequality constraints on functions of the state and control variables, and bounds on the values of the control variables. Necessary conditions for optimal control of the control problem are derived. In the iterative procedure, no prior information is required regarding the sequence of constrained and unconstrained arcs and the inequality constraints which are on their boundaries along a specific constrained arc of the optimal trajectory. All this information will be determined within the iterative procedure using some necessary conditions for optimal control. The ability of the proposed algorithm to solve practical problems is demonstrated by its application to several variations of two problems, one of which is a common manipulator problem in industry where transportation of open vessels of liquid is to be performed in a specified period of time. It is shown that the proposed quasilinearization algorithm is an effective tool in deriving optimal control policies for a common type of manipulator operation in industry.     

A perturbation method for control systems with multiple switch points

We consider a system arising from an application of the Maximum Principle to a free endpoint trajectory optimization problem arising in control. The system involves a small parameter, and has the property that the optimal control associated with the reduced problem (epsilon=0) moves on and off the boundary of the control region a finite number of times. We show how a technique involving a nonlinear chgnge of independent variables can be used to obtain uniformly valid parameter expansions for the solution of the full problem for ε small, and we establish conditions on the Hamiltonian under which this procedure may be carried out.