Finite Horizon Minimax Optimal Control of Stochastic Partially Observed Time Varying Uncertain Systems

We consider a linear-quadratic problem of minimax optimal control for stochastic uncertain control systems with output measurement. The uncertainty in the system satisfies a stochastic integral quadratic constraint. To convert the constrained optimization problem into an unconstrained one, a special S-procedure is applied. The resulting unconstrained game-type optimization problem is then converted into a risk-sensitive stochastic control problem with an exponential-of-integral cost functional. This is achieved via a certain duality relation between stochastic dynamic games and risk-sensitive stochastic control. The solution of the risk-sensitive stochastic control problem in terms of a pair of differential matrix Riccati equations is then used to establish a minimax optimal control law for the original uncertain system with uncertainty subject to the stochastic integral quadratic constraint. Date received: May 13, 1997. Date revised: March 18, 1998.     

Risk-sensitive control and dynamic games for partially observeddiscrete-time nonlinear systems

Solves a finite-horizon partially observed risk-sensitive stochastic optimal control problem for discrete-time nonlinear systems and obtains small noise and small risk limits. The small noise limit is interpreted as a deterministic partially observed dynamic game, and new insights into the optimal solution of such game problems are obtained. Both the risk-sensitive stochastic control problem and the deterministic dynamic game problem are solved using information states, dynamic programming, and associated separated policies. A certainty equivalence principle is also discussed. The authors' results have implications for the nonlinear robust stabilization problem. The small risk limit is a standard partially observed risk-neutral stochastic optimal control problem     

General finite-dimensional risk-sensitive problems and small noiselimits

For a risk-sensitive, partially observed stochastic control problem, the modified Zakai equation includes an extra term related to the exponential running cost. The finite-dimensional solutions of this modified Zakai equation are obtained. These are analogs of the Kalman and Benes filters. The small noise limits of the finite-dimensional risk-sensitive problems are then obtained. These lead to differential games with deterministic disturbances     

Risk-Sensitive and Robust Escape Control for Degenerate Diffusion Processes

This paper considers the problem of controlling a possibly degenerate small noise diffusion so as to prevent it from leaving a prescribed set. The criterion of interest is a risk-sensitive version of the mean escape time criterion. Using a general representation formula, this criterion is expressed as the upper value of a stochastic differential game. It is shown that in the small noise limit this upper value converges to the value of an associated deterministic differential game. Our approach differs from standard PDE approaches in a number of ways. For example, the upper game representation allows one to relate directly the prelimit and the limit controls and, in fact, strategies that are nearly maximizing for the robust problem can be used to define nearly minimizing controls for the risk-sensitive control problem for sufficiently small ε>0. The result provides a canonical example of the use of variational representations in connecting risk-sensitive and robust control. Date received: November 21, 1998. Date revised: June 20, 1999.     

H∞ Control of Discrete-Time Systems With Multiple Input Delays

This paper is concerned with the finite horizon H_infin full-information control for discrete-time systems with multiple control and exogenous input delays. We first establish a duality between the H_infin full-information control and the H₂ smoothing of a stochastic backward system in Krein space. Like the duality between the linear quadratic regulation (LQR) of linear systems without delays and the Kalman filtering, the established duality allows us to address complicated multiple input delay problems in a simple way. Indeed, by applying innovation analysis and standard projection in Krein space, in this paper we derive conditions under which the H_infin full-information control is solvable. An explicit controller is constructed in terms of two standard Riccati difference equations of the same order as the original plant (ignoring the delays). As special cases, solutions to the H_infin state feedback control problem for systems with delays only in control inputs and the H_infin control with preview are obtained. An example is given to demonstrate the effectiveness of the proposed H_infin control design     

Partially observed non-linear risk-sensitive optimal stopping control for non-linear discrete-time systems

In this paper we introduce and solve the partially observed optimal stopping non-linear risk-sensitive stochastic control problem for discrete-time non-linear systems. The presented results are closely related to previous results for finite horizon partially observed risk-sensitive stochastic control problem. An information state approach is used and a new (three-way) separation principle established that leads to a forward dynamic programming equation and a backward dynamic programming inequality equation (both infinite dimensional). A verification theorem is given that establishes the optimal control and optimal stopping time. The risk-neutral optimal stopping stochastic control problem is also discussed.     

The stochastic maximum principle for optimal control problems of delay systems involving continuous and impulse controls

This paper is concerned with a Pontryagin’s maximum principle for stochastic optimal control problems of delay systems with random coefficients involving both continuous and impulse controls. This kind of control problems is motivated by some interesting phenomena arising from economics and finance. We establish a necessary maximum principle and a sufficient verification theorem by virtue of the duality and the convex analysis. To explain the theoretical results, we apply them to a production and consumption choice problem.     

Quantum Risk-Sensitive Estimation and Robustness

This paper studies a quantum risk-sensitive estimation problem and investigates robustness properties of the filter. This is a direct extension to the quantum case of analogous classical results. All investigations are based on a discrete approximation model of the quantum system under consideration. This allows us to study the problem in a simple mathematical setting. We close the paper with some examples that demonstrate the robustness of the risk-sensitive estimator.     

Optimal short-term scheduling of large-scale power systems

This paper is concerned with the longstanding problem of optimal unit commitment in an electric power system. We follow the traditional formulation of this problem which gives rise to a large-scale, dynamic, mixed-integer programming problem. We describe a solution methodology based on duality, Lagrangian relaxation, and nondifferentiable optimization that has two unique features. First, computational requirements typically grow only linearly with the number of generating units. Second, the duality gap decreases in relative terms as the number of units increases, and as a result our algorithm tends to actually perform better for problems of large size. This allows for the first time consistently reliable solution of large practical problems involving several hundreds of units within realistic time constraints. Aside from the unit commitment problem, this methodology, is applicable to a broad class of large-scale dynamic scheduling and resource allocation problems involving integer variables.     

The equivalence between infinite-horizon optimal control ofstochastic systems with exponential-of-integral performance index andstochastic differential games

A new method, based on the theory of large deviations from the invariant measure, is introduced for the analysis of stochastic systems with an infinite-horizon exponential-of-integral performance index. It is shown that the infinite-horizon optimal exponential-of-integral stochastic control problem is equivalent to a stationary stochastic differential game for an auxiliary system. As an application of the developed technique, the infinite-horizon risk-sensitive LQG problem is analyzed for both the completely observed and partially observed case     

Controller design with multiple objectives

In this paper we study multi-objective control problems that give rise to equivalent convex optimization problems. We develop a uniform treatment of such problems by showing their equivalence to linear programming problems with equality constraints and an appropriate positive cone. We present some specialized results on duality theory, and we apply them to the study of three multi-objective control problems: the optimal l₁ control with time-domain constraints on the response to some fixed input, the mixed H₂/l₁ -control problem, and the l₁ control with magnitude constraint on the frequency response. What makes these problems complicated is that they are often equivalent to infinite-dimensional optimization problems. The characterization of the duality relationship between the primal and dual problem allows us to derive several results. These results establish connections with special convex problems (linear programming or linear matrix inequality problems), uncover finite-dimensional structures in the optimal solution, when possible, and provide finite-dimensional approximations to any degree of accuracy when the problem does not appear to have a finite-dimensional structure. To illustrate the theory and highlight its potential, several numerical examples are presented     

Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games

In this paper we consider a finite horizon, nonlinear, stochastic, risk-sensitive optimal control problem with complete state information, and show that it is equivalent to a stochastic differential game. Risk-sensitivity and small noise parameters are introduced, and the limits are analyzed as these parameters tend to zero. First-order expansions are obtained which show that the risk-sensitive controller consists of a standard deterministic controller, plus terms due to stochastic and game-theoretic methods of controller design. The results of this paper relate to the design of robust controllers for nonlinear systems.Research supported in part by the 1990 Summer Faculty Research Fellowship, University of Kentucky.     

Risk-sensitive control for a class of homing problems

A result of Lefebvre [Lefebvre, M. (2001). A different class of homing problems. Systems & Control Letters 42, 347-352] is here extended to a two-dimensional homing problem with a risk-sensitive cost criterion. It is shown that the optimal control is given explicitly, and moreover, the optimal value function has a simple probabilistic representation associated with a backward stochastic differential equation with a random terminal time.     

Duality and lower bounds in optimal stochastic control

A dual problem is proposed for an optimal stochastic control problem whose dynamical system is described by a linear stochastic differential equation. Relationships between the extreme values of the original and dual problems are discussed and two duality theorems are obtained The dual problem furnishes lower bounds for the extreme value of the original problem.     

Design of satisfaction output feedback controls for stochastic nonlinear systems under quadratic tracking risk-sensitive index

In this paper, the design problem of satisfaction output feedback controls for stochastic nonlinear systems in strict feedback form under long-term tracking risk-sensitive index is investigated. The index function adopted here is of quadratic form usually encountered in practice, rather than of quartic one used to beg the essential difficulty on controller design and performance analysis of the closed-loop systems. For any given risk-sensitive parameter and desired index value, by using the integrator backstepping method, an output feedback control is constructively designed so that the closed-loop system is bounded in probability and the risk-sensitive index is upper bounded by the desired value.     

The role of information state and adjoint in relating nonlinearoutput feedback risk-sensitive control and dynamic games

This paper employs logarithmic transformations to establish relations between continuous-time nonlinear partially observable risk-sensitive control problems and analogous output feedback dynamic games. The first logarithmic transformation is introduced to relate the stochastic information state to a deterministic information state. The second logarithmic transformation is applied to the risk-sensitive cost function using the Laplace-Varadhan lemma. In the small noise limit, this cost function is shown to be logarithmically equivalent to the cost function of an analogous dynamic game     

Risk-sensitive control of Markov jump linear systems: Caveats and difficulties

In this technical note, we revisit the risk-sensitive optimal control problem for Markov jump linear systems (MJLSs). We first demonstrate the inherent difficulty in solving the risk-sensitive optimal control problem even if the system is linear and the cost function is quadratic. This is due to the nonlinear nature of the coupled set of Hamilton-Jacobi-Bellman (HJB) equations, stemming from the presence of the jump process. It thus follows that the standard quadratic form of the value function with a set of coupled Riccati differential equations cannot be a candidate solution to the coupled HJB equations. We subsequently show that there is no equivalence relationship between the problems of risk-sensitive control and H ^∞ control of MJLSs, which are shown to be equivalent in the absence of any jumps. Finally, we show that there does not exist a large deviation limit as well as a risk-neutral limit of the risk-sensitive optimal control problem due to the presence of a nonlinear coupling term in the HJB equations.

     

A risk-sensitive control dual approach to a large deviations control problem

We consider a control problem of an ergodic process where the objective is to maximize over long term the probability to overperform a given level. This is formulated as a large deviations control problem for which the standard dynamic programming methods may not be applied directly. We solve this problem by adopting a duality approach leading to a risk-sensitive ergodic control problem. In a continuous-time diffusion setting, we state a verification theorem in terms of partial differential equations for this dual problem. We then turn back to the primal problem by means of large deviations techniques. We derive the optimal rate function and nearly optimal controls for the large deviations optimization problem. Finally, explicit solutions are provided in a linear-quadratic case.     

Robust output feedback stabilization via risk-sensitive control

We consider a problem of robust linear quadratic Gaussian (LQG) control for discrete-time stochastic uncertain systems with partial state measurements. For a finite-horizon case, the problem was recently introduced by Petersen et al. (IEEE Trans. Automat. Control 45 (2000) 398). In this paper, an infinite horizon extension of the results of Petersen et al. (IEEE Trans. Automat. Control 45 (2000) 398) is discussed. We show that for a broad class of uncertain systems under consideration, a controller constructed in terms of the solution to a specially parameterized risk-sensitive stochastic control problem absolutely stabilizes the stochastic uncertain system.     

Dynamic hedging of basket options under proportional transaction costs using receding horizon control

In this article, we develop a semi-definite programming-based receding horizon control approach to the problem of dynamic hedging of European basket call options under proportional transaction costs. The hedging problem for a European call option is formulated as a finite horizon constrained stochastic control problem. This allows us to develop a receding horizon control approach that repeatedly solves semi-definite programmes on-line in order to dynamically hedge. This approach is competitive with Black–Scholes delta hedging in the one-dimensional case with no transaction costs, but it also applies to multi-dimensional options such as basket options, and can include transaction costs. We illustrate its effectiveness through a numerical example involving an option on a basket of five stocks.