Reduction of future information required for optimal control of dynamic systems: a pseudostochastic model

This note develops a pseudostochastic model for optimal control of dynamic systems over a given planning horizon. The obtained results reflect the extent to which reduction of future information impacts upon the performance and optimal control of dynamic systems. The main results indicate that, when using only partial information for determining optimal control, the performance of the dynamic system is almost identical to that when using full information. When ignoring the information expected beyond the planning horizon, a significant performance loss and a possible violation of feasibility of the optimal control can occur.     

Shortest path stochastic control for hybrid electric vehicles

When a hybrid electric vehicle (HEV) is certified for emissions and fuel economy, its power management system must be charge sustaining over the drive cycle, meaning that the battery state of charge (SOC) must be at least as high at the end of the test as it was at the beginning of the test. During the test cycle, the power management system is free to vary the battery SOC so as to minimize a weighted combination of fuel consumption and exhaust emissions. This paper argues that shortest path stochastic dynamic programming (SP‐SDP) offers a more natural formulation of the optimal control problem associated with the design of the power management system because it allows deviations of battery SOC from a desired setpoint to be penalized only at key off. This method is illustrated on a parallel hybrid electric truck model that had previously been analyzed using infinite‐horizon stochastic dynamic programming with discounted future cost. Both formulations of the optimization problem yield a time‐invariant causal state‐feedback controller that can be directly implemented on the vehicle. The advantages of the shortest path formulation include that a single tuning parameter is needed to trade off fuel economy and emissions versus battery SOC deviation, as compared with two parameters in the discounted, infinite‐horizon case, and for the same level of complexity as a discounted future‐cost controller, the shortest‐path controller demonstrates better fuel and emission minimization while also achieving better SOC control when the vehicle is turned off. Linear programming is used to solve both stochastic dynamic programs. Copyright © 2007 John Wiley & Sons, Ltd.     

Optimal production control in a discrete manufacturing system withunreliable machines and random demands

We consider a production control problem in a manufacturing system with a failure-prone machine and a stochastic demand. The objective is to minimize a discounted inventory holding and backlog cost over an infinite planning horizon. The optimal production control of continuous, stochastic manufacturing systems with a failure-prone machine and a constant demand has been considered in Akella and Kumar (1986). However, the problem of optimal production control for discrete stochastic manufacturing systems with uncertain demands remains open. In this paper, we investigate a case where the exogenous demand forms a homogeneous Poisson flow. Primarily, we show that the optimal production control for such a system is of the threshold control type. In addition, the explicit form of production control policy and the objective functions are provided. Numerical examples are included to demonstrate the results obtained in the paper and to compare with the one in Akella and Kumar     

Shrinking‐horizon dynamic programming

We describe a heuristic control policy for a general finite‐horizon stochastic control problem, which can be used when the current process disturbance is not conditionally independent of the previous disturbances, given the current state. At each time step, we approximate the distribution of future disturbances (conditioned on what has been observed) by a product distribution with the same marginals. We then carry out dynamic programming (DP), using this modified future disturbance distribution, to find an optimal policy, and in particular, the optimal current action. We then execute only the optimal current action. At the next step, we update the conditional distribution, and repeat the process, this time with a horizon reduced by one step. (This explains the name ‘shrinking‐horizon dynamic programming’). We explain how the method can be thought of as an extension of model predictive control, and illustrate our method on two variations on a revenue management problem. Copyright © 2010 John Wiley & Sons, Ltd.     

H∞ tracking of linear continuous‐time systems with stochastic uncertainties and preview

The problem of finite‐horizon H_∞ tracking for linear continuous time‐invariant systems with stochastic parameter uncertainties is investigated for both, the state‐feedback and the output‐feedback control problems. We consider three tracking patterns depending on the nature of the reference signal i.e. whether it is perfectly known in advance, measured on line or previewed in a fixed time‐interval ahead. The stochastic uncertainties appear in both the dynamic and measurement matrices of the system. In the state‐feedback case, for each of the above three cases a game theory approach is applied where, given a specific reference signal, the controller plays against nature which chooses the initial condition and the energy‐bounded disturbance. The problems are solved using the expected value of the standard performance index over the stochastic parameters, where, in the state‐feedback case, necessary and sufficient conditions are found for the existence of a saddle‐point equilibrium. The corresponding infinite‐horizon time‐invariant tracking problem is also solved for the latter case, where a dissipativity approach is considered. The output‐feedback control problem is solved as a max–min problem for the three tracking patterns, where necessary and sufficient condition are obtained for the solution. The theory developed is demonstrated by a simple example where we compare our solution with an alternative solution which models the tracking signal as a disturbance. Copyright © 2004 John Wiley & Sons, Ltd.     

Optimal finite-horizon production control in a defect-prone environment

In this note, we consider a single-machine, single-part-type production system, operating in a defect-prone environment. It is assumed that there is a random yield proportion of nondefective parts, with known probability distribution. Over each production cycle, it is assumed that there is a single realization of the yield random variable. Furthermore, it is assumed that the system is operated under a periodic-review policy. Thus, the particular realization of the yield proportion cannot be determined prior to the end of the production horizon. The optimal production control, that minimizes a linear combination of expected surplus and shortage costs over the planning horizon is shown to be piecewise constant, and the appropriate production levels and control break-points are determined as functions of the yield rate distribution.     

Receding horizon finite memory controls for output feedback controls of state-space systems

In this paper, a new type of control, called a receding horizon finite memory control (RHFMC) or a model predictive finite memory control, is proposed as an optimal output feedback control for stochastic state-space systems. Constraints such as linearity, finite memory structure, and unbiasedness from the optimal state feedback control are required in advance, and in addition, the performance criterion of quadratic cost is required. Constraints for the input and the state are not assumed in this paper. The RHFMC is obtained directly by minimizing the performance criterion for stochastic state-space systems with the previous constraints. It is shown that the RHFMC can be separated into a receding horizon control and a finite-impulse response filter. The stability of the RHFMC is investigated. The validity of the proposed RHFMC is illustrated by a numerical example.     

Stochastic Optimal Design for Unknown Linear Discrete‐Time System Zero‐Sum Games in Input‐Output form Under Communication Constraints

In this paper, stochastic optimal strategy for unknown linear discrete‐time system quadratic zero‐sum games in input‐output form with communication imperfections such as network‐induced delays and packet losses, otherwise referred to as networked control system (NCS) zero‐sum games, relating to the H_∞ optimal control problem is solved in a forward‐in‐time manner. First, the linear discrete‐time zero sum state space representation is transformed into a linear NCS in the state space form after incorporating random delays and packet losses and then into the input‐output form. Subsequently, the stochastic optimal approach, referred to as adaptive dynamic programming (ADP), is introduced which estimates the cost or value function to solve the infinite horizon optimal regulation of unknown linear NCS quadratic zero‐sum games in the presence of communication imperfections. The optimal control and worst case disturbance inputs are derived based on the estimated value function in the absence of state measurements. An update law for tuning the unknown parameters of the value function estimator is derived and Lyapunov theory is used to show that all signals are asymptotically stable (AS) and that the estimated control and disturbance signals converge to optimal control and worst case disturbances, respectively. Simulation results are included to verify the theoretical claims.     

H∞ prediction and unconstrained H∞ predictive control: multi‐input–multi‐output case

Through the combination of the sequential spectral factorization and the coprime factorization, a k‐step ahead MIMO H_∞ (cumulative minimax) predictor is derived which is stable for the unstable noise model. This predictor and the modified internal model of the reference signal are embedded into the H_∞ optimization framework, yielding a single degree of freedom multi‐input–multi‐output H_∞ predictive controller that provides stochastic disturbance rejection and asymptotic tracking of the reference signals described by the internal model. It is shown that for a plant/disturbance model, that represents a large class of systems, the inclusion of the H_∞ predictor into the H_∞ control algorithm introduces a performance/robustness tuning knob: an increase of the prediction horizon enforces a more conservative control effort and, correspondingly, results in deterioration of the transient and the steady‐state (tracking error variance) performance, but guarantees large robustness margin, while the decrease of the prediction horizon results in a more aggressive control signal and better transient and steady‐state performance, but smaller robustness margin. Copyright © 2001 John Wiley & Sons, Ltd.     

Robust stochastic maximum principle for multi-model worst case optimization

This paper develops a version of the robust maximum principle applied to the minimax Mayer problem formulated for stochastic differential equations with a control-dependent diffusion term. The parametric families of first and second order adjoint stochastic processes are introduced to construct the corresponding Hamiltonian formalism. The Hamiltonian function used for the construction of the robust optimal control is shown to be equal to the sum of the standard stochastic Hamiltonians corresponding to each possible value of the parameter. The cost function is defined on a finite horizon and contains the mathematical expectation of a terminal term. A terminal condition, given by a vector function, is also considered. The optimal control strategies, adapted for available information, for the wide class of multi-model systems given by a stochastic differential equation with parameters from a given finite set are constructed. This problem belongs to the class of minimax stochastic optimization problems. The proof is based on the recent results obtained for deterministic minimax Mayer problem by Boltyanski and Poznyak as well as on the results of Zhou and of Yong and Zhou, obtained for stochastic maximum principle for non-linear stochastic systems with a single-valued parameter. Two illustrative examples, dealing with production planning and reinsurance-dividend management, conclude this study.     

Efficient management of interconnected power systems: A game-theoretic approach

The optimal management over a one year planning horizon, of two interconnected hydro-thermal power systems is considered. The optimal production in each system is modelled as a stochastic control problem whose solution is searched in a particular class of control strategies. The efficient exchange of energy between the two systems and its pricing are viewed as a cooperative game and the Nash-Harsanyi bargaining solution is characterized. Various information structures for the exchange and price strategies are discussed and it is shown that, in all cases, the price strategy is equivalent to the definition of a compensatory side payment which equalizes the advantages accruing to each of the two players with respect to a status quo situation where no interconnection is available. A numerical illustration based on a typical European power system is presented to assess the potential gain when using a closed loop exchange strategy instead of an open loop one.     

On stochastic dynamic programming for solving large-scale planning problems under uncertainty

The stochastic dynamic programming approach outlined here, makes use of the scenario tree in a back-to-front scheme. The multi-period stochastic problems, related to the subtrees whose root nodes are the starting nodes (i.e., scenario groups), are solved at each given stage along the time horizon. Each subproblem considers the effect of the stochasticity of the uncertain parameters from the periods of the given stage, by using curves that estimate the expected future value (EFV) of the objective function. Each subproblem is solved for a set of reference levels of the variables that also have nonzero elements in any of the previous stages besides the given stage. An appropriate sensitivity analysis of the objective function for each reference level of the linking variables allows us to estimate the EFV curves applicable to the scenario groups from the previous stages, until the curves for the first stage have been computed. An application of the scheme to the problem of production planning with logical constraints is presented. The aim of the problem consists of obtaining the planning of tactical production over the scenarios along the time horizon. The expected total cost is minimized to satisfy the product demand. Some computational experience is reported. The proposed approach compares favorably with a state-of-the-art optimization engine in instances on a very large scale.     

A simulation‐based algorithm for optimal pricing policy under demand uncertainty

We propose a simulation‐based algorithm for computing the optimal pricing policy for a product under uncertain demand dynamics. We consider a parameterized stochastic differential equation (SDE) model for the uncertain demand dynamics of the product over the planning horizon. In particular, we consider a dynamic model that is an extension of the Bass model. The performance of our algorithm is compared to that of a myopic pricing policy and is shown to give better results. Two significant advantages with our algorithm are as follows: (a) it does not require information on the system model parameters if the SDE system state is known via either a simulation device or real data, and (b) as it works efficiently even for high‐dimensional parameters, it uses the efficient smoothed functional gradient estimator.     

Scheduling one-part-type serial manufacturing system under periodic demand: a solvable case

The paper studies one-part type, multiple-stage production system with periodic demands. A buffer of infinite capacity is placed after each machine. Inventory flow through buffers is controlled by machine production rates. The objective is to find a cyclic production rate, which minimizes all inventory-related expenses over an infinite planning horizon. With the aid of the maximum principle, optimal production policies are derived and the continuous-time scheduling problem is reduced to a discrete timing problem. As a result, a polynomial-time algorithm is suggested to calculate the optimal production rate. A numerical example is used to illustrate the algorithm.Scope and purposeNumerical and heuristic approaches have been suggested for production control of automated-serial-manufacturing systems. These approaches try to derive production control policies that would minimize overall costs related to inventory, backlog, and production. The quality of these approaches is often difficult to assess, and they can be time-consuming to implement. Therefore, increasing attention has been directed to optimal control policies of production systems that can be derived precisely and quickly. This paper addresses a special case of the production system manufacturing a single product type to meet a periodic demand. Given a certain assumption on cost relationship, we derive a fast and simple scheduling algorithm that calculates the optimal policy.     

Medium term scheduling of a hydro-thermal system using stochastic model predictive control

A multistage stochastic programming formulation is presented for monthly production planning of a hydro-thermal system. Stochasticity from variations in water reservoir inflows and fluctuations in demand of electric energy are considered explicitly. The problem can be solved efficiently via Nested Benders Decomposition. The solution is implemented in a model predictive control setup and performance of this control technique is demonstrated in simulations. Tuning parameters, such as prediction horizon and shape of the stochastic programming tree are identified and their effects are analyzed.     

Energy-efficient receding horizon trajectory planning of high-speed trains using real-time traffic information

Optimal trajectory planning of high-speed trains (HSTs) aims to obtain such speed curves that guarantee safety, punctuality, comfort and energy-saving of the train. In this paper, a new shrinking horizon model predictive control (MPC) algorithm is proposed to plan the optimal trajectories of HSTs using real-time traffic information. The nonlinear longitudinal dynamics of HSTs are used to predict the future behaviors of the train and describe variable slopes and variable speed limitations based on real-time traffic information. Then optimal trajectory planning of HSTs is formulated as the shrinking horizon optimal control problem with the consideration of safety, punctuality, comfort and energy consumption. According to the real-time position and running time of the train, the shrinking horizon is updated to ensure the recursive feasibility of the optimization problem. The optimal speed curve of the train is computed by online solving the optimization problem with the Radau Pseudo-spectral method (RPM). Simulation results demonstrate that the proposed method can satisfy the requirements of energy efficiency and punctuality of the train.     

Control of a hybrid stochastic system

We consider in this paper a continuous-time stochastic hybrid control system with a finite time horizon. The objective is to minimize a linear function of the expected state trajectory. The state evolves according to a linear dynamics. However, the parameters of the state evolution equation may change at discrete times according to a controlled Markov chain which has finite state and action spaces. We use a procedure similar in form to the maximum principle; this determines a control strategy which is asymptotically optimal as the number of transitions during the finite time horizon grows to infinity.     

Asymptotic analysis of nonlinear stochastic risk-sensitive control and differential games

In this paper we consider a finite horizon, nonlinear, stochastic, risk-sensitive optimal control problem with complete state information, and show that it is equivalent to a stochastic differential game. Risk-sensitivity and small noise parameters are introduced, and the limits are analyzed as these parameters tend to zero. First-order expansions are obtained which show that the risk-sensitive controller consists of a standard deterministic controller, plus terms due to stochastic and game-theoretic methods of controller design. The results of this paper relate to the design of robust controllers for nonlinear systems.Research supported in part by the 1990 Summer Faculty Research Fellowship, University of Kentucky.     

需求随机的变质性产品生产计划模型

研究了能力约束的有限计划展望期生产计划问题,各周期的需求随机,库存产品存在变质且变质率为常数。建立了问题的期望值模型,目标函数为极小化生产准备成本、生产成本、库存成本的期望值。提出了随机模拟、遗传算法和启发式算法相结合的求解算法。用数值实例对模型和算法进行了验证,优化结果表明模型和算法是有效的。