Linear convex stochastic control problems over an infinite horizon

A stochastic control problem over an infinite horizon which involves a linear system and a convex cost functional is analyzed. We prove the convergence of the dynamic programming algorithm associated with the problem, and we show the existence of a stationary Borel measurable optimal control law. The approach used illustrates how results on infinite time reachability [1] can be used for the analysis of dynamic programming algorithms over an infinite horizon subject to state constraints.     

A Policy Improvement Method in Constrained Stochastic Dynamic Programming

This note presents a formal method of improving a given base-policy such that the performance of the resulting policy is no worse than that of the base-policy at all states in constrained stochastic dynamic programming. We consider finite horizon and discounted infinite horizon cases. The improvement method induces a policy iteration-type algorithm that converges to a local optimal policy.     

An interactive fuzzy satisficing method based on variance minimization under expectation constraints for multiobjective stochastic linear programming problems

In this paper, we focus on multiobjective linear programming problems involving random variable coefficients in objective functions and constraints. Using the concept of chance constrained conditions, such multiobjective stochastic linear programming problems are transformed into deterministic ones based on the variance minimization model under expectation constraints. After introducing fuzzy goals to reflect the ambiguity of the decision maker??s judgements for objective functions, we propose an interactive fuzzy satisficing method to derive a satisficing solution for them as a fusion of the stochastic programming and the fuzzy one. The application of the proposed method to an illustrative numerical example shows its usefulness.     

Detecting planning horizons in deterministic infinite horizon optimal control

A deterministic infinite horizon optimal control problem with discounted cost is investigated. Under some assumptions we give a method to detect a planning horizon which is guaranteed to exist. When forward dynamic programming can be applied, the procedure does not require extra computations. An inventory control example is presented.     

基于动态规划的资源受限随机工序调度

为解决资源受限条件下的随机工序调度问题,该文提出一种基于离散随机动态系统描述的加工时间离散随机分布且同时具有不兼容和多种可更新资源约束的资源受限项目调度模型,使得在满足资源约束和工序约束的前提下,总的平均加工时间最短。该系统研究了动态规划算法求解该问题的方法。通过实例,验证了该方法的有效性和可行性。     

Probabilistic tubes in linear stochastic model predictive control

This paper considers constrained control of linear systems with additive and multiplicative stochastic uncertainty and linear input/state constraints. Both hard and soft constraints are considered, and bounds are imposed on the probability of soft constraint violation. Assuming the plant parameters to be finitely supported, a method of constraint handling is proposed in which a sequence of tubes, corresponding to a sequence of confidence levels on the predicted future plant state, is constructed online around nominal state trajectories. A set of linear constraints is derived by imposing bounds on the probability of constraint violation at each point on an infinite prediction horizon through constraints on one-step-ahead predictions. A guarantee of the recursive feasibility of the online optimization ensures that the closed loop system trajectories satisfy both the hard and probabilistic soft constraints. The approach is illustrated by a numerical example.     

INTERACTIVE FUZZY PROGRAMMING BASED ON FRACTILE CRITERION OPTIMIZATION MODEL FOR TWO-LEVEL STOCHASTIC LINEAR PROGRAMMING PROBLEMS

In this article, we focus on two-level linear programming problems involving random variable coefficients in objective functions and constraints. Following the concept of chance constrained programming, the two-level stochastic linear programming problems are transformed into deterministic ones based on the fractile criterion optimization model. After introducing fuzzy goals for objective functions, interactive fuzzy programming to derive a satisfactory solution for decision makers is presented as a fusion of a stochastic approach and a fuzzy one. An illustrative numerical example is provided to demonstrate the feasibility of the proposed method.     

Newton-conjugate gradient （CG） augmented Lagrangian method for path constrained dynamic process optimization

In this paper, a Newton-conjugate gradient (CG) augmented Lagrangian method is proposed for solving the path constrained dynamic process optimization problems. The path constraints are simplified as a single final time constraint by using a novel constraint aggregation function. Then, a control vector parameterization (CVP) approach is applied to convert the constraints simplified dynamic optimization problem into a nonlinear programming (NLP) problem with inequality constraints. By constructing an augmented Lagrangian function, the inequality constraints are introduced into the augmented objective function, and a box constrained NLP problem is generated. Then, a linear search Newton-CG approach, also known as truncated Newton (TN) approach, is applied to solve the problem. By constructing the Hamiltonian functions of objective and constraint functions, two adjoint systems are generated to calculate the gradients which are needed in the process of NLP solution. Simulation examples demonstrate the effectiveness of the algorithm.     

Shortest path stochastic control for hybrid electric vehicles

When a hybrid electric vehicle (HEV) is certified for emissions and fuel economy, its power management system must be charge sustaining over the drive cycle, meaning that the battery state of charge (SOC) must be at least as high at the end of the test as it was at the beginning of the test. During the test cycle, the power management system is free to vary the battery SOC so as to minimize a weighted combination of fuel consumption and exhaust emissions. This paper argues that shortest path stochastic dynamic programming (SP‐SDP) offers a more natural formulation of the optimal control problem associated with the design of the power management system because it allows deviations of battery SOC from a desired setpoint to be penalized only at key off. This method is illustrated on a parallel hybrid electric truck model that had previously been analyzed using infinite‐horizon stochastic dynamic programming with discounted future cost. Both formulations of the optimization problem yield a time‐invariant causal state‐feedback controller that can be directly implemented on the vehicle. The advantages of the shortest path formulation include that a single tuning parameter is needed to trade off fuel economy and emissions versus battery SOC deviation, as compared with two parameters in the discounted, infinite‐horizon case, and for the same level of complexity as a discounted future‐cost controller, the shortest‐path controller demonstrates better fuel and emission minimization while also achieving better SOC control when the vehicle is turned off. Linear programming is used to solve both stochastic dynamic programs. Copyright © 2007 John Wiley & Sons, Ltd.     

Stochastic Receding Horizon Control of Constrained Linear Systems With State and Control Multiplicative Noise

We develop a receding horizon control approach to stochastic linear systems with control and state multiplicative noise that also contain constraints. Our receding horizon formulation is based upon an on-line optimization that utilizes open-loop plus linear feedback and is solved as a semi-definite programming problem. We also provide a characterization of stability, performance, and constraint satisfaction properties of the receding horizon controlled system under a specific choice of terminal weight and terminal constraint. A simple numerical example is used to illustrate the approach.      

Quadratic control of stochastic hybrid systems with renewal transitions

We study the quadratic control of a class of stochastic hybrid systems with linear continuous dynamics for which the lengths of time that the system stays in each mode are independent random variables with given probability distribution functions. We derive a condition for finding the optimal feedback policy that minimizes a discounted infinite horizon cost. We show that the optimal cost is the solution to a set of differential equations with unknown boundary conditions. Furthermore, we provide a recursive algorithm for computing the optimal cost and the optimal feedback policy. The applicability of our result is illustrated through a numerical example, motivated by stochastic gene regulation in biology.     

Preventive maintenance and replacement scheduling for repairable and maintainable systems using dynamic programming

This paper presents mathematical models and a solution approach to determine the optimal preventive maintenance schedules for a repairable and maintainable series system of components with an increasing rate of occurrence of failure (ROCOF). The maintenance planning horizon has been divided into discrete and equally-sized periods and in each period, three possible actions for each component (maintain it, replace it, or do nothing) have been considered. The optimal decisions for each component in each period are investigated such that the objectives and the requirements of the system can be achieved. In particular, the cases of minimizing total cost subject to a constraint on system reliability, and maximizing system reliability subject to a budgetary constraint on overall cost have been modeled. As the optimization methodology, dynamic programming combined with branch-and-bound method is utilized and the effectiveness of the approach is presented through the use of a numerical example. Such a modeling approach should be useful for maintenance planners and engineers tasked with the problem of developing recommended maintenance plans for complex systems of components.     

Performance bounds and suboptimal policies for linear stochastic control via LMIs

In a recent paper, the authors showed how to compute performance bounds for infinite‐horizon stochastic control problems with linear system dynamics and arbitrary constraints, objective, and noise distribution. In this paper, we extend these results to the finite‐horizon case, with asymmetric costs and constraint sets. In addition, we derive our bounds using a new method, where we relax the Bellman equation to an inequality. The method is based on bounding the objective with a general quadratic function, and using linear matrix inequalities (LMIs) and semidefinite programming (SDP) to optimize the bound. The resulting LMIs are more complicated than in the previous paper (which only used quadratic forms) but this extension allows us to obtain good bounds for problems with substantial asymmetry, such as supply chain problems. The method also yields very good suboptimal control policies, using control‐Lyapunov methods. Copyright © 2010 John Wiley & Sons, Ltd.     

Convexity and convex approximations of discrete-time stochastic control problems with constraints

We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost subject to probabilistic constraints. We study the convexity of a finite-horizon optimization problem in the case where the control policies are affine functions of the disturbance input. We propose an expectation-based method for the convex approximation of probabilistic constraints with polytopic constraint function, and a Linear Matrix Inequality (LMI) method for the convex approximation of probabilistic constraints with ellipsoidal constraint function. Finally, we introduce a class of convex expectation-type constraints that provide tractable approximations of the so-called integrated chance constraints. Performance of these methods and of existing convex approximation methods for probabilistic constraints is compared on a numerical example.     

Solution of the input-constrained LQR problem using dynamic programming

The input-constrained LQR problem is addressed in this paper; i.e., the problem of finding the optimal control law for a linear system such that a quadratic cost functional is minimised over a horizon of length N subject to the satisfaction of input constraints. A global solution (i.e., valid in the entire state space) for this problem, and for arbitrary horizon N, is derived analytically by using dynamic programming. The scalar input case is considered in this paper. Solutions to this problem (and to more general problems: state constraints, multiple inputs) have been reported recently in the literature, for example, approaches that use the geometric structure of the underlying quadratic programming problem and approaches that use multi-parametric quadratic programming techniques. The solution by dynamic programming proposed in the present paper coincides with the ones obtained by the aforementioned approaches. However, being derived using a different approach that exploits the dynamic nature of the constrained optimisation problem to obtain an analytical solution, the present result complements the previous methods and reveals additional insights into the intrinsic structure of the optimal solution.     

Fast algorithm for a constrained infinite horizon LQ problem

This paper presents an efficient algorithmic solution to the infinite horizon linear quadratic optimal control problem for a discrete-time SISO plant subject to bound constraints on a scalar variable. The solution to the corresponding quadratic programming problem is based on the active set method and on dynamic programming. It is shown that the optimal solution can be updated after inclusion or removal of an active constraint by a simple procedure requiring in the order of kn operations, n being the system order and k the time at which the constraint is included or removed.     

Genetic based fuzzy goal programming for multiobjective chance constrained programming problems with continuous random variables

Solution procedure consisting of fuzzy goal programming and stochastic simulation-based genetic algorithm is presented, in this article, to solve multiobjective chance constrained programming problems with continuous random variables in the objective functions and in chance constraints. The fuzzy goal programming formulation of the problem is developed first using the stochastic simulation-based genetic algorithm. Without deriving the deterministic equivalent, chance constraints are used within the genetic process and their feasibilities are checked by the stochastic simulation technique. The problem is then reduced to an ordinary chance constrained programming problem. Again using the stochastic simulation-based genetic algorithm, the highest membership value of each of the membership goal is achieved and thereby the most satisfactory solution is obtained. The proposed procedure is illustrated by a numerical example.     

A forward method for optimal stochastic nonlinear and adaptivecontrol

A computational approach is taken to solve the optimal partially observed nonlinear stochastic control problem. The approach is to systematically solve the stochastic dynamic programming equations forward in time, using a nested stochastic approximation technique. Although computationally intensive, this provides a straightforward numerical solution for this class of problems and provides an alternative to the usual `curse of dimensionality' associated with solving the dynamic programming equation backwards in time. In particular, the `curse' is seen to take a new form, where the amount of computation depends on the amount of uncertainty in the problem and the length of the horizon. As a matter of more practical interest, it is shown that the cost degrades monotonically as the complexity of the algorithm is reduced. This provides a strategy for suboptimal control with clear performance/computation trade-offs. A numerical study focusing on a generic optimal stochastic adaptive control example is included to demonstrate the feasibility of the method     

Stochastic model predictive control for probabilistically constrained Markovian jump linear systems with additive disturbance

Model predictive control (MPC) for Markovian jump linear systems with probabilistic constraints has received much attention in recent years. However, in existing results, the disturbance is usually assumed with infinite support, which is not considered reasonable in real applications. Thus, by considering random additive disturbance with finite support, this paper is devoted to a systematic approach to stochastic MPC for Markovian jump linear systems with probabilistic constraints. The adopted MPC law is parameterized by a mode‐dependent feedback control law superimposed with a perturbation generated by a dynamic controller. Probabilistic constraints can be guaranteed by confining the augmented system state to a maximal admissible set. Then, the MPC algorithm is given in the form of linearly constrained quadratic programming problems by optimizing the infinite sum of derivation of the stage cost from its steady‐state value. The proposed algorithm is proved to be recursively feasible and to guarantee constraints satisfaction, and the closed‐loop long‐run average cost is not more than that of the unconstrained closed‐loop system with static feedback. Finally, when adopting the optimal feedback gains in the predictive control law, the resulting MPC algorithm has been proved to converge in the mean square sense to the optimal control. A numerical example is given to verify the efficiency of the proposed results.     

A robust constrained model predictive control scheme for norm‐bounded uncertain systems with partial state measurements

A robust model predictive control scheme for a class of constrained norm‐bounded uncertain discrete‐time linear systems is developed under the hypothesis that only partial state measurements are available for feedback. The proposed strategy involves a two‐phase procedure. Initialization phase is devoted to determining an admissible, though not optimal, linear memoryless controller capable to formally address the input rate constraint; then, during on‐line phase, predictive capabilities complement the designed controller by means of N steps free control actions in a receding horizon fashion. These additive control actions are obtained by solving semidefinite programming problems subject to linear matrix inequalities constraints. As computational burden grows linearly with the control horizon length, an example is developed to show the effectiveness of the proposed approach for realistic control problems: the design of a flight control law for a flexible unmanned over‐actuated aircraft, where the states of the flexibility dynamics are not measurable, is discussed, and a numerical implementation of the controller within a nonlinear simulation environment testifies the validity of the proposed approach and the possibility to implement the algorithm on an onboard computer.