Discrete stochastic system linear output control with respect to a quadratic criterion

An optimization problem with the quality criterion that differs from the conventional one by some modifications is given. The state of the system is described by a second-order unobservable random sequence, while the observations are described by an optimizable noisy linear output. The fact that the unobservable state of the system is independent of the control is substantiated by interpreting the statement as the optimization problem for resource allocation for the system as opposed to the conventional statement that suggests controlling its state. A variant of the practical substantiation of the statement involved, i.e., optimizing operation of a software system, is proposed. A dynamic programming method is used to solve the problem. The optimal strategy is found as a linear combination of the output and predictions of the state up to the optimization horizon. Since the optimal strategy is computationally laborious, the possibility to simplify it and apply a locally optimal strategy is discussed.     

One-parameter optimal correction problem for the trajectory of an aerial vehicle with respect to the probability criterion

The correction problem of an aerial vehicle trajectory is considered. The mathematical model of the correction process is represented by a scalar stochastic control system with a probability terminal performance index. The system’s state variable is the predicted miss of a single parameter of the aerial vehicle. It is assumed that the complete information about the state variable is available. The aim of the correction is to maximize the probability that the terminal miss does not exceed the prescribed level. The execution errors of the designed correction impulse are distributed uniformly. Using dynamic programming, a procedure for the optimization of corrections of the aerial vehicle trajectory with respect to the probability performance index is developed, and this procedure is used to solve the one-parameter optimal correction problem for the aerial vehicle for the case of N time steps. The resulting optimal control is compared to the known optimal controls with respect to other performance indices.     

Dual criterion stochastic optimal control problem for robustness improvement

The solution of the dual criterion linear quadratic stochastic optimal control problem is obtained by following a Wiener type of solution procedure. A stabilizing solution is guaranteed by parameterizing the controller using the Desoer fractional representation approach. The dual criterion includes sensitivity and complementary sensitivity weighting terms which provide a means of varying the robustness characteristics of the multivariable system.     

Design of optimal strategies in the problems of discrete system control by the probabilistic criterion

A modification was proposed for the relations of the method of dynamic programming in the problems of optimal stochastic control of the discrete systems by the probabilistic performance criterion. It enabled one to simplify the process of finding the optimal Markov strategy and obtain a suboptimal solution. Its efficiency was verified by the examples of maneuver optimization of the stationary satellite in the neighborhood of a geostationary orbit. An explicit form of the optimal control for the bilinear system with probabilistic terminal criterion was determined using the results obtained.     

Infinite horizon optimal control of linear discrete time systems with stochastic parameters

The infinite horizon optimal control problem is considered in the general case of linear discrete time systems and quadratic criteria, both with stochastic parameters which are independent with respect to time. A stronger stabilizability property and a weaker observability property than usual for deterministic systems are introduced. It is shown that the infinite horizon problem has a solution if the system has the first property. If in addition the problem has the second property the solution is unique and the control system is stable in the mean square sense. A simple necessary and sufficient condition, explicit in the system matrices, is given for the system to have the stronger stabilizability property. This condition also holds for deterministic systems to be stabilizable in the usual sense. The stronger stabilizability and weaker observability properties coincide with the usual ones if the parameters are deterministic.     

On the infinite time solution to state-constrained stochastic optimal control problems

A method is presented for solving the infinite time Hamilton-Jacobi-Bellman (HJB) equation for certain state-constrained stochastic problems. The HJB equation is reformulated as an eigenvalue problem, such that the principal eigenvalue corresponds to the expected cost per unit time, and the corresponding eigenfunction gives the value function (up to an additive constant) for the optimal control policy. The eigenvalue problem is linear and hence there are fast numerical methods available for finding the solution.     

On the optimal control of a stochastic system with discontinuous sample paths

This paper deals with the optimal control of a stochastic n-order system. It is assumed that the system is subjected to two different kinds of perturbations. The first kind of perturbation is represented by a vector of independent standard Wiener processes and the second kind by a vector of a generalized type of Poisson process. By applying the calculus of variations necessary conditions on the optimal controls are derived. These conditions are given by a pair of coupled non-linear partial integro-differential equations, A stochastic second-order system is given, as a test case, and a numerical method for the computation of its optimal controls is suggested. The efficiency and applicability of this method ore demonstrated with examples.     

Application of stochastic optimal control to a hydrofoil boat problem

In this paper the solution of a stochastic optimal control problem described by linear equations of motion and a nonquadratic performance index is presented. The theory is then applied to the dynamics of a single-foil and a hydrofoil boat flying on rough water. The random disturbances caused by sea waves are represented as the response of an auxiliary system to a white noise input. The control objective is formulated as an integral performance index containing a quadratic acceleration term and a nonquadratic term of the submergence deviation of the foil from calm water submergence. The stochastic version of the maximum principle is used in the formulation of a feedback control law. The Riccati equations and the feedback gains associated with a nonquadratic performance index are non-linear functions of the state and auxiliary state variables. These equations are integrated forward with the state equations for the steady-state solution of the problem. The controller for a nonquadratic performance index contains computing elements which perform the integration of the Riccati equations to generate the instantaneous values of the feedback gains. The effect of a nonquadratic penalty on the submergence deviation and the effect of a nonquadratic control penalty on the response of the system are investigated. A comparison between an optimal nonlinear control law and a suboptimal linear control law is presented.     

On the existence of optimal solutions for the linear system quadratic cost problem

The linear systemdot{x} = Ax +buwith the quadratic cost functionintmin{0}max{infty}(x'Hx+u^{2})dtis considered. The following equivalent conditions are shown to be necessary and sufficient for the existence of a solution to the optimization problem: 1) the existence of a positive definite solution for the algebraic matrix Riccati equation; 2) the existence of a vectorksuch that the matrixA-bk'is asymptotically stables and 3) the cancellations in the vector(sI-A)^{-1}bare only of stable factors.     

On a result in stochastic optimal control

In a recent paper an equivalence was demonstrated between a certain stochastic optimal control problem and a linear-quadratic game. In this note it is pointed out that a more general equivalence exists which has an interesting form.     

Hybrid method for a general optimal sensor scheduling problem in discrete time

In this paper, we consider a general class of optimal sensor scheduling problems in discrete time. There are N₁ sensors available for acquiring data so as to estimate the needed but unknown signal. Only N₂ out of the N₁ sensors can be turned on at any moment, while different weights can be assigned to different sensors. This problem is formulated as a discrete time deterministic optimal control problem involving both discrete and continuous valued controls. A computational method is developed for solving this discrete time deterministic optimal control problem based on a branch and bound method in conjunction with a gradient-based method. The branch and bound method is used to determine the optimal schedule of sensors, where a sequence of lower bound dynamic systems is introduced so as to provide effective lower bounds for the construction of the branching rules. Each of the branches is an optimal weight vector assignment problem and a gradient-based method is developed for solving this optimal control problem. For illustration, two numerical examples are solved.     

On optimal control of mean-field stochastic systems driven by Teugels martingales via derivative with respect to measures

ABSTRACT

This paper deals with partial information stochastic optimal control problem for general controlled mean-field systems driven by Teugels martingales associated with some Lévy process having moments of all orders, and an independent Brownian motion. The coefficients of the system depend on the state of the solution process as well as of its probability law and the control variable. We establish a set of necessary conditions in the form of Pontryagin maximum principle for the optimal control. We also give additional conditions, under which the necessary optimality conditions turn out to be sufficient. The proof of our result is based on the derivative with respect to the probability law by applying Lions derivatives and a corresponding Itô formula. As an application, conditional mean-variance portfolio selection problem in incomplete market, where the system is governed by some Gamma process is studied to illustrate our theoretical results.     

Infinite time optimal control for a class of stochastic systems

Optimization results developed by the author (Brandeberry and Wu 1970) for a class of stochastic regulator problems will be extended from the finite time interval to the infinite time interval. Conditions are given for the existence and stability of the infinite constant feedback of the system state; necessary conditions will also be obtained for the inverse problem (i.e. conditions for a linear constant control law to be optimal for some cost functional).     

On the two-stage problem of linear stochastic programming with quantile criterion and discrete distribution of the random parameters

Consideration was given to the two-stage problem of stochastic linear programming with a discrete distribution of the random parameter vector. The property of continuity of the quantile function in strategy was proved, the sufficient conditions for existence of solution were formulated, and an algorithm to determine the guaranteeing solution was constructed on the basis of the confidence method and the duality theorem. A deterministic equivalent of the considered problem in the form of a linear programming problem was given for the scalar case.     

A problem of optimal control of a stochastic sheet

The problem of optimal control of the solution of a stochastic differential equation with a fractional Wiener sheet is investigated.     

Maximum principle for the stochastic optimal control problem with delay and application

In this paper, we consider an optimal control problem for the stochastic system described by stochastic differential equations with delay. We obtain the maximum principle for the optimal control of this problem by virtue of the duality method and the anticipated backward stochastic differential equations. Our results can be applied to a production and consumption choice problem. The explicit optimal consumption rate is obtained.     

Sufficiency conditions for existence of an optimal feedback control in stochastic mechanics

In the framework of stochastic mechanics, the following problem is considered: in a set of admissible feedback controls v, with range inE ⁿ , find one minimizing the expectationE _sx { _s ^T L(t, (t), (t, (t)))dt + W _T ((T))} for all (s, x) [0,T) E ⁿ , whereL(t, x, ) = (/12)m ² – U(t, x) is the classical action integrand and is an-dimensional diffusion process in the weak sense, (see Bensoussan, 1982) with drift and diffusion coefficientD constant > 0.W _T andU are given real functions. Sufficiency conditions for the existence of such an optimal feedback control are given. Dedicated to George Leitmann Recommended by G.J. Olsder Presented at the Third Workshop on Control Mechanics in honor of George Leitmann, January 22–24, 1990, University of Southern California, Los Angeles, California (USA).