Nonstationary discrete-time deterministic and stochastic control systems with infinite horizon

This article is about nonstationary nonlinear discrete-time deterministic and stochastic control systems with Borel state and control spaces, possibly noncompact control constraint sets, and unbounded costs. The control problem is to minimise an infinite-horizon total cost performance index. Using dynamic programming arguments we show that, under suitable assumptions, the optimal cost functions satisfy optimality equations, which in turn give a procedure to find optimal control policies.     

切换线性系统的聚合优化

分析了切换线性系统的稳定性和优化问题,提出了一个Armijo步长优化的共轭梯度算法来寻找适当代价函数下的优化切换时间点集.为了确保优化切换路径是可压缩的,提出了代价函数需要满足的受限表达式.设计了几种优化分段状态反馈切换律来搜索聚合系统的最优切换路径,同时这些切换路径就是对应原始切换线性系统的次优化切换路径.最后,一个实例演示了不同切换律下的切换策略和优化代价.     

Stabilization and optimization of switched linear systems

In this paper, we establish the equivalence among switched convergency, asymptotic stabilizability, and exponential stabilizability for force-free switched linear systems, and discuss the implication to the infinite-time horizon optimal switching problem. We show that, for a general cost function under mild assumptions, the finiteness of the optimal cost is equivalent to the asymptotic stabilizability of the switched linear system. Finally, we prove the equality between the optimal costs for the switched system and for the relaxed differential inclusion.     

Mathematical programs with complementarity constraints in the context of inverse optimal control for locomotion

In this paper an inverse optimal control problem in the form of a mathematical program with complementarity constraints (MPCC) is considered and numerical experiences are discussed. The inverse optimal control problem arises in the context of human navigation where the body is modelled as a dynamical system and it is assumed that the motions are optimally controlled with respect to an unknown cost function. The goal of the inversion is now to find a cost function within a given parametrized family of candidate cost functions such that the corresponding optimal motion minimizes the deviation from given data. MPCCs are known to be a challenging class of optimization problems typically violating all standard constraint qualifications (CQs). We show that under certain assumptions the resulting MPCC fulfills CQs for MPCCs being the basis for theory on MPCC optimality conditions and consequently for numerical solution techniques. Finally, numerical results are presented for the discretized inverse optimal control problem of locomotion using different solution techniques based on relaxation and lifting.     

On the formulation of cost functions for torque-optimized control of rigid bodies

In the context of controlling the attitude of a rigid body, this communique uses recent results on representations of torques (moments) to establish cost functions. The resulting cost functions are both properly invariant under whatever choice of Euler angles is used to parameterize the rotation of the rigid body and have transparent physical interpretations. The function is related to existing works in geometric control theory and applications of optimal control theory to biomechanical systems.     

Infinitesimal Perturbation Analysis and Optimization for Make-to-Stock Manufacturing Systems Based on Stochastic Fluid Models

In this paper we study Make-To-Stock manufacturing systems and seek on-line algorithms for determining optimal or near optimal buffer capacities (hedging points) that balance inventory against stockout costs. Using a zStochastic Fluid Model (SFM), we derive sample derivatives (sensitivities) which, under very weak structural assumptions on the defining demand and service processes, are shown to be unbiased estimators of the sensitivities of a cost function with respect to these capacities. When evaluated based on the sample path of discrete-part systems, we show that these estimators are greatly simplified. Thus, they can be easily implemented and evaluated on line. Though the implementation on discrete-part systems does not necessarily preserve the unbiasedness property, simulation results show that stochastic approximation algorithms that use such estimates do converge to optimal or near optimal hedging points. Supported in part by the National Science Foundation under Grants EEC-0088073 and DMI-0330171, by AFOSR under contract F49620-01-0056, and by ARO under grant DAAD19-01-0610.     

Vehicle cruise: Improved fuel economy by periodic control

It is shown that time-dependent periodic control can improve the fuel economy of vehicles in cruise. The time-dependent controls considered are relaxed steady-state (RSS) control, quasi-steady-state (QSS) control and quasi-relaxed steady-state (QRSS) control. Examples are given which show that QRSS control may give better performance than either RSS or QSS control. Properties of optimal cost functions, dependent on the minimum required average speed, are derived. The possibility or impossibility of improved performance through the use of QRSS, QSS and RSS control is investigated in terms of assumptions on the vehicle drag and fuel-consumption functions.     

Global inverse optimal stabilization of stochastic nonholonomic systems

Optimality has not been addressed in existing works on control of (stochastic) nonholonomic systems. This paper presents a design of optimal controllers with respect to a meaningful cost function to globally asymptotically stabilize (in probability) nonholonomic systems affine in stochastic disturbances. The design is based on the Lyapunov direct method, the backstepping technique, and the inverse optimal control design. A class of Lyapunov functions, which are not required to be as nonlinearly strong as quadratic or quartic, is proposed for the control design. Thus, these Lyapunov functions can be applied to design of controllers for underactuated (stochastic) mechanical systems, which are usually required Lyapunov functions of a nonlinearly weak form. The proposed control design is illustrated on a kinematic cart, of which wheel velocities are perturbed by stochastic noise.     

Convex parametric piecewise quadratic optimization: Theory and algorithms

In this paper we study the problem of parametric minimization of convex piecewise quadratic functions. Our study provides a unifying framework for convex parametric quadratic and linear programs. Furthermore, it extends parametric optimization algorithms to problems with piecewise quadratic cost functions, paving the way for new applications of parametric optimization in explicit dynamic programming and optimal control with quadratic stage cost.     

Average cost optimal policies for Markov control processes with Borel state space and unbounded costs

We show the existence of average cost optimal stationary policies for Markov control processes with Borel state space and unbounded costs per stage, under a set of assumptions recently introduced by L.I. Sennott (1989) for control processes with countable state space and finite control sets.     

Safe Adaptive Switching Control: Stability and Convergence

A formal theoretical explanation of the model-mismatch instability problem associated with certain adaptive control design schemes is proposed, and a solution is provided. To address the model-mismatch problem, a primary task of adaptive control is formulated as finding an asymptotically optimal, stabilizing controller, given the feasibility of adaptive control problem. A class of data-driven cost functions called cost-detectable is introduced that detect evidence of instability without reference to prior plant models or plant assumptions. The problem of designing adaptive systems that are robustly immune to mismatch instability problems is thus placed in a setting of a standard optimization problem. We call the result safe adaptive control because it robustly achieves adaptive stabilization goals whenever feasible, without prior assumptions on the plant model and, hence, without the risk of model-mismatch instability. The result improves the robustness of previous results in hysteresis switching control, both for discrete and for continuously-parameterized candidate controller sets. Examples are provided.     

Neural approximations for infinite-horizon optimal control of nonlinear stochastic systems.

A feedback control law is proposed that drives the controlled vector v(t) of a discrete-time dynamic system to track a reference v(t)* over an infinite time horizon, while minimizing a given cost function. The behavior of v(t)* over time is completely unpredictable, Random noises act on the dynamic system and the state observation channel, which may be nonlinear. It is assumed that all such random vectors are mutually independent, and that their probability density functions are known. So general a non-LQG optimal control problem is very difficult to solve. The proposed solution is based on three main approximating assumptions: 1) the problem is stated in a receding-horizon framework where v(t)* is assumed to remain constant within a shifting-time window; 2) the control law is assigned a given structure (that of a multilayer feedforward neural net) in which a finite number of parameters have to be determined so as to minimize the cost function; and 3) the control law is given a limited memory, which prevents the amount of data to be stored from increasing over time. Errors resulting from the second and third assumptions are discussed, Due to the very general assumptions under which the control law is derived, we are not able to report stability results. However, simulation results show that the proposed method may constitute an effective tool for solving, to a sufficient degree of accuracy, a wide class of control problems traditionally regarded as difficult ones. An example of freeway traffic optimal control is given.     

Input-to-state stability of robust receding horizon control with an expected value cost

This paper is concerned with the stability of a class of receding horizon control (RHC) laws for constrained linear discrete-time systems subject to bounded state disturbances and convex state and input constraints. The paper considers the class of finite horizon feedback control policies parameterized as affine functions of the system state, calculation of which can be shown to be tractable via a convex reparameterization. When minimizing the expected value of a finite horizon quadratic cost, we show that the value function is convex. When solving this optimal control problem at each time step and implementing the result in a receding horizon fashion, we provide sufficient conditions under which the closed-loop system is input-to-state stable (ISS).     

ITSE optimal controller design and achievable performance bounds

This article proposes a methodology to design optimal controllers and to compute achievable performance bounds in the control of linear, stable discrete-time SISO plants. The performance is measured using a time-weighted cost function (ITSE) of the tracking error for a step reference or a step output disturbance, combined with a measure of the energy of the incremental control. The proposed methodology relies on the use of a function space product that, for stable systems, can be defined in both the frequency and the time-domains. The solution technique requires the expansion of stable functions in orthogonal basis. An analytical expression is found for the computation of the optimal coefficients of the expansion. Numerical examples are included to illustrate the results.     

Singular ergodic control for multidimensional Gaussian processes

A multidimensional Wiener process is controlled by an additive process of bounded variation. A convex nonnegative function measures the cost associated with the position of the state process, and the cost of controlling is proportional to the displacement induced. We minimize a limiting time-average expected (ergodic) criterion. Under reasonable assumptions, we prove that the optimal discounted cost converges to the optimal ergodic cost. Moreover, under some additional conditions there exists a convex Lipschitz continuous function solution to the corresponding Hamilton-Jacobi-Bellman equation which provides an optimal stationary feedback control.Research supported in part by NSF Grant DMS-8702236.Research supported in part by Grant AFOSR-88-D183.     

Quadratic control of stochastic hybrid systems with renewal transitions

We study the quadratic control of a class of stochastic hybrid systems with linear continuous dynamics for which the lengths of time that the system stays in each mode are independent random variables with given probability distribution functions. We derive a condition for finding the optimal feedback policy that minimizes a discounted infinite horizon cost. We show that the optimal cost is the solution to a set of differential equations with unknown boundary conditions. Furthermore, we provide a recursive algorithm for computing the optimal cost and the optimal feedback policy. The applicability of our result is illustrated through a numerical example, motivated by stochastic gene regulation in biology.     

A counterexample on the optimality equation in Markov decision chains with the average cost criterion

We consider Markov decision processes with denumerable state space and finite control sets; the performance index of a control policy is a long-run expected average cost criterion and the cost function is bounded below. For these models, the existence of average optimal stationary policies was recently established in [11] under very general assumptions. Such a result was obtained via an optimality inequality. Here, we use a simple example to prove that the conditions in [11] do not imply the existence of a solution to the average cost optimality equation.     

Event-driven optimization-based control of hybrid systems with integral continuous-time dynamics

In this paper we introduce a class of continuous-time hybrid dynamical systems called integral continuous-time hybrid automata (icHA) for which we propose an event-driven optimization-based control strategy. Events include both external actions applied to the system and changes of continuous dynamics (mode switches). The icHA formalism subsumes a number of hybrid dynamical systems with practical interest, e.g., linear hybrid automata. Different cost functions, including minimum-time and minimum-effort criteria, and constraints are examined in the event-driven optimal control formulation. This is translated into a finite-dimensional mixed-integer optimization problem, in which the event instants and the corresponding values of the control input are the optimization variables. As a consequence, the proposed approach has the advantage of automatically adjusting the attention of the controller to the frequency of event occurrence in the hybrid process. A receding horizon control scheme exploiting the event-based optimal control formulation is proposed as a feedback control strategy and proved to ensure either finite-time or asymptotic convergence of the closed-loop.     

需求和退货与供应中断相关环境下库存控制

需求和退货与供应中断相关会引发库存剧烈波动,从而导致库存控制非常困难.在采用Markov调制Lévy过程描述库存水平变化条件下,利用水平穿越法、更新过程和鞅理论确定循环期望时间和费用函数,据此构建系统长程平均费用率模型.在此基础上,通过仿真实验研究需求(退货)与供应中断相关度对最优库存控制策略的影响,并分析不同中断和退货类型下最优控制策略的变化,从而为该环境下有效管理库存提供新启示.     

Suboptimal control of linear systems with delays in state and input by orthonormal basis

This paper presents a method for finding the optimal control of linear systems with delays in state and input and the quadratic cost functional using an orthonormal basis for square integrable functions. The state and control variables and their delays are expanded in an orthonormal basis with unknown coefficients. Using operational matrices of integration, delay and product, the relation between coefficients of variables is provided. Then, necessary condition of optimality is driven as a linear system of algebraic equations in terms of the unknown coefficients of state and control variables. As an application of this method, the linear Legendre multiwavelets as an orthonormal basis for L ²[0, 1] are used. Two time-delayed optimal control problems are approximated to establish the usefulness of the method.