A stochastic optimal feedback control problem with random-sized jumps

An optimal feedback control problem for a partially observed linear system with noise of fixed-sized jumps occurring at random times driven by a Poisson process is extended to include noise with random-sized jumps. The control structure is appropriately modified to compensate for the mean behavior of the system jumps via an additional deterministic term. Editor: N.U. Ahmed     

A simple dual control problem with an analytical solution

A stochastic control problem for which the optimal dual control law can be calculated analytically is given. The system is a four state Markov chain with transition probabilities that depend on the control variable. The performance of the optimal dual control law and of some suboptimal control laws are calculated and compared.     

Congestion control as a stochastic control problem with action delays

We consider the design of explicit rate-based congestion control for high-speed communication networks and show that this can be formulated as a stochastic control problem where the controls of different users enter the system dynamics with different delays. We discuss the existence, derivation and the structure of the optimal controller, as well as of suboptimal controllers of the certainty-equivalent type — a terminology that is precisely defined in the paper for the specific context of the congestion control problem considered. We consider, in particular, two certainty-equivalent controllers which are easy to implement, and show that they are stabilizing, i.e., they lead to bounded infinite-horizon average cost, and stable queue dynamics. Further, these controllers perform well in simulations.     

An optimal stochastic control problem with observation cost

The problem of simultaneously determining an optimal control strategy and an optimal observation strategy for a linear system is considered. Quadratic costs on state and control and an "on-off" type observation cost are assumed. Dynamic programming is used to obtain a solution. An example is provided that shows some interesting relations between the optimal observation strategy and various system parameters.     

A solvable stochastic control problem in hyperbolic three space

A stochastic optimal control problem in hyperbolic three space is described and explicitly solved. The solution is obtained by finding a smooth solution to the Hamilton-Jacobi or dynamic programming equation.     

A problem of optimal control of a stochastic sheet

The problem of optimal control of the solution of a stochastic differential equation with a fractional Wiener sheet is investigated.     

A nonlinear control law for a stochastic infinite time problem

The expectation of a particular class of nonquadratic performance criterion involving even powers of the state variables up to sixth order is minimized, over an infinite horizon, subject to a linear stochastic system. The process noise is composed of both additive white noise and state dependent white noise processes. The resulting controller is composed of a linear and cubic function of the state. Furthermore, this controller depends upon the noise variances of both the additive and state dependent noise processes. For partially observable systems with no state dependent noise, similar results as the completely observable system are implied by the separation theorem. For state dependent noise alone, a stochastic Lyapunov function is obtained from which simple probability bounds for the trajectory to exit from a given region of the state space are determined.     

A stochastic production planning problem with nonlinear cost

Most production planning models are deterministic and often assume a linear relation between production volume and production cost. In this paper, we investigate a production planning problem in a steel production process considering the energy consumption cost which is a nonlinear function of the production quantity. Due to the uncertain environment, the production demands are stochastic. Taking a scenario-based approach to express the stochastic demands according to the knowledge of planners on the demand distributions, we formulate the stochastic production planning problem as a mixed integer nonlinear programming (MINLP) model.Approximated with the piecewise linear functions, the MINLP model is transformed into a mixed integer linear programming model. The approximation error can be improved by adjusting the linearization ranges repeatedly. Based on the piecewise linearization, a stepwise Lagrangian relaxation (SLR) heuristic for the problem is proposed where variable splitting is introduced during Lagrangian relaxation (LR). After decomposition, one subproblem is solved by linear programming and the other is solved by an effective polynomial time algorithm. The SLR heuristic is tested on a large set of problem instances and the results show that the algorithm generates solutions very close to optimums in an acceptable time. The impact of demand uncertainty on the solution is studied by a computational discussion on scenario generation.     

On a reliability problem by stochastic control methods

In the paper a problem of renewal of a nonhomogeneous Markov chain with incomplete information given by a point process is treated. A long run average cost is studied. The article applies the ‘crossing level’ method given by Davis. An example of exponentially distributed time-to-failure is given.     

A stochastic fleet composition problem

In this paper, we consider the problem of forming a new vehicle fleet, consisting of multiple vehicle types, to cater for uncertain future requirements. The problem is to choose the number of vehicles of each type to purchase so that the total expected cost of operating the fleet is minimized. The total expected cost includes fixed and variable costs associated with the fleet, as well as hiring costs that are incurred whenever vehicle requirements exceed fleet capacity. We develop a novel algorithm, which combines dynamic programming and the golden section method, for determining the optimal fleet composition. Numerical results show that this algorithm is highly effective, and takes just seconds to solve large-scale problems involving hundreds of different vehicle types.     

The drilling problem: A stochastic modeling and control example in manufacturing

A machining economics problem is considered where feed rate selection and tool replacement policies are to be determined. A new stochastic model for tool wear, called a diffusion-threshold model, is proposed. This tool wear model allows the machining economics problem to be formulated as a stochastic optimal control problem incorporating measurement feedback of tool wear. Two types of control policies are described. One is a traditional machining economics policy and the other utilizes tool wear feedback and allows on-line decision making. The optimal policy is described for both types. An example problem based on actual data is worked out that compares the two approaches and demonstrates the utility of information feedback and on-line control.     

The optimum stochastic control for a problem with random cost incursion times

A sequential stochastic control problem is considered in which the performance criterion cost is incremented periodically at random times. Although these cost incursion times are unknown, they are governed by a known prior probability density function. The optimum Bayesian control decision strategy is obtained by solving the stochastic dynamic programming equation. As anticipated, the solution exhibits a type of separation principle.     

Maximum principle for the stochastic optimal control problem with delay and application

In this paper, we consider an optimal control problem for the stochastic system described by stochastic differential equations with delay. We obtain the maximum principle for the optimal control of this problem by virtue of the duality method and the anticipated backward stochastic differential equations. Our results can be applied to a production and consumption choice problem. The explicit optimal consumption rate is obtained.     

Convex stochastic programmes with simple recourse

Convex stocliastic programmes with simple recourse are one of the few classes of stochastic programmes which are amenable to a full analysis. Here we exploit the structure of this programme using generalized geometric programming to reduce it to simple deterministic form. The resulting non-linear programme is readily solvable by existing techniques.     

A newsboy problem with a simple reservation arrangement

Newsboy problems have always been an important issue in inventory management. A number of methods have been proposed to solve such problems based on different objectives or considerations. Different from the existing studies, this paper presents a newsboy model with a simple reservation arrangement by introducing the willingness rate, represented as the function of the discount rate, into the models. Mathematical models are developed, and the solution procedure is derived for determining the optimal discount rate and the optimal order quantity. Through the numerical example, we demonstrated the varied profits yielded from the models considering the reservation arrangement, depending on the number of consumers who accept the reservation policy. The profits derived are greater than those from the classical newsboy model, due to the consideration of the reservation.     

Robust optimal control for minimax stochastic linear quadratic problem

The robust maximum principle applied to the minimax linear quadratic problem is derived for stochastic differential equations containing a control-dependent diffusion term. The parametric families of the first and second order adjoint stochastic processes are obtained to construct the corresponding Hamiltonian formalism. The Hamiltonian function used for the construction of the robust optimal control is shown to be equal to the sum of the standard stochastic Hamiltonians corresponding to each value of the uncertain parameter from a given finite set. The cost function is considered on a finite horizon (contains the mathematical expectation of both an integral and a terminal term) and on an infinite one (a time-averaged losses function). These problems belong to the class of minimax stochastic optimization problems. It is shown that the construction of the minimax optimal controller can be reduced to an optimization problem on a finitedimensional simplex and consists in the analysis of the dependence of Riccati equation solution on the weight parameters to be found.     

The zero divisor problem of multivariable stochastic adaptive control

Stochastic adaptive minimum variance control algorithms require a division by a function of a recursively computed parameter estimate at each instant of time. In order that the analysis of these algorithms is valid, zero divisions must be events of probability zero. This property is established for the stochastic gradient adaptive control algorithm under the condition that the initial state of the system and all finite segments of its random disturbance process have a joint distribution which is absolutely continuous with respect to Lebesgue measure. This result is deduced from the following general result established in this paper: a non-constant rational function of a finite set of random variables {x₁},x_n} is absolutely continuous with respect to Lebesgue measure if the joint distribution function of {x₁,…,x_n} has this property.     

Dual criterion stochastic optimal control problem for robustness improvement

The solution of the dual criterion linear quadratic stochastic optimal control problem is obtained by following a Wiener type of solution procedure. A stabilizing solution is guaranteed by parameterizing the controller using the Desoer fractional representation approach. The dual criterion includes sensitivity and complementary sensitivity weighting terms which provide a means of varying the robustness characteristics of the multivariable system.     

Decentralized,stochastic control for a static LQ problem

An original problem statement for the optimal control laws design is presented. A large scale system composed of M linear static subsystems with an interaction and quadratic performance index is considered. A two-level hierarchical control structure is assumed, in which a coordinator and local controllers have access to different information. The so-called elastic constraint (Gessing 1985) is used for coordination. For the problem the possibility of partial decomposition of calculations and a decentralization of the control, as well as an analytical form of the optimal laws are obtained. The influence of the particular subsystem on the control quality is investigated.