A new approach to the analysis of discrete event dynamic systems

We present a new, time domain approach to the study of discrete event dynamical systems (DEDS), typified by queueing networks and production systems. A general state-space representation is developed and perturbation analysis is carried out. Observation of a single sample realization of such a system can be used to predict behavior over other sample realizations, when some parameter is perturbed, without having to make additional observations. Conditions under which this is always possible are investigated and explicit results for some special cases are included.     

Analysis of input-to-state stability for discrete time nonlinear systems via dynamic programming

The input-to-state stability (ISS) property for systems with disturbances has received considerable attention over the past decade or so, with many applications and characterizations reported in the literature. The main purpose of this paper is to present analysis results for ISS that utilize dynamic programming techniques to characterize minimal ISS gains and transient bounds. These characterizations naturally lead to computable necessary and sufficient conditions for ISS. Our results make a connection between ISS and optimization problems in nonlinear dissipative systems theory (including L₂-gain analysis and nonlinear H_∞ theory). As such, the results presented address an obvious gap in the literature.     

GA-based discrete dynamic programming approach for scheduling inFMS environments

The paper presents a new genetic algorithm (GA)-based discrete dynamic programming (DDP) approach for generating static schedules in a flexible manufacturing system (FMS) environment. This GA-DDP approach adopts a sequence-dependent schedule generation strategy, where a GA is employed to generate feasible job sequences and a series of discrete dynamic programs are constructed to generate legal schedules for a given sequence of jobs. In formulating the GA, different performance criteria could be easily included. The developed DDF algorithm is capable of identifying locally optimized partial schedules and shares the computation efficiency of dynamic programming. The algorithm is designed In such a way that it does not suffer from the state explosion problem inherent in pure dynamic programming approaches in FMS scheduling. Numerical examples are reported to illustrate the approach.     

A control synthesis approach for time discrete event systems

In this paper, we introduce a control synthesis method for discrete event systems whose behavior is dependent on explicit values of time. Our goal is to control the occurrence dates of the controllable events so that the functioning of the system respects given specifications. The system to be controlled is modeled by a time Petri net. In a previous work we proposed a systematic method to build the timed automaton which models the exact behavior of a time Petri net. Furthermore, the forbidden behaviors of the system are modeled by forbidden timed automaton locations. This paper focuses on the control synthesis method, which consists in computing new firing conditions for the timed automaton transitions so that the forbidden locations are no longer reachable.     

A state prediction scheme for discrete time nonlinear dynamic systems

A state prediction scheme is proposed for discrete time nonlinear dynamic systems with non-Gaussian disturbance and observation noises. This scheme is based upon quantization, multiple hypothesis testing, and dynamic programming. Dynamic models of the proposed scheme are as general as dynamic models of particle predictors, whereas the nonlinear models of the extended Kalman (EK) predictor are linear with respect to the disturbance and observation noises. The performance of the proposed scheme is compared with both the EK predictor and sampling importance resampling (SIR) particle predictor. Monte Carlo simulations have shown that the performances of the proposed scheme, EK predictor, and SIR particle predictor are all model-dependent, that is, one performs better than the others for a given example. Some examples, for which the proposed scheme performs better than the others do, are also given in the paper.     

一种间歇反应过程迭代动态规划方法

随着工业过程对降低产品成本、改进产品质量、满足安全要求和环境规范,间歇反应过程的优化变得越来越重要.本文因此给出了一种有效的基于随机选点的间歇反应过程迭代动态规划算法,并给出了算法实现的详细步骤,能够有效实现间歇反应过程中温度、浓度等变量的动态优化问题.所述的迭代动态规划算法通过调节分段数P和离散点数(N和M)可以有效的避免计算量激增的问题,具有稳定可靠、易寻找到全局最优解的优点.以典型的间歇反应动态优化问题作为实例进行了研究,并与国际上公开报道结果进行了详细的比较研究,结果表明了所述方法的可靠有效性.     

A dynamic programming approach to trajectory estimation

An iterative equation based on dynamic programming for finding the most likely trajectory of a dynamic system observed through a noisy measurement system is presented; the procedure can be applied to nonlinear systems with non-Gaussian noise. It differs from the recently developed Bayesian estimation procedure in that the most likely estimate of the entire trajectory up to the present time, rather than of the present state only, is generated. It is shown that the two procedures in general yield different estimates of the present state; however, in the case of linear systems with Gaussian noise, both procedures reduce to the Kalman-Bucy filter. Illustrative examples are presented, and the present procedure is compared with the Bayesian procedure and with other estimation techniques in terms of computational requirements and applicability.     

A dynamic programming approach to sequential pattern recognition

This paper presents the dynamic programming approach to the design of optimal pattern recognition systems when the costs of feature measurements describing the pattern samples are of considerable importance. A multistage or sequential pattern classifier which requires, on the average, a substantially smaller number of feature measurements than that required by an equally reliable nonsequential classifier is defined and constructed through the method of recursive optimization. Two methods of reducing the dimensionality in computation are presented for the cases where the observed feature measurements are 1) statistically independent, and 2) Markov dependent. Both models, in general, provide a ready solution to the optimal sequential classification problem. A generalization in the design of optimal classifiers capable of selecting a best sequence of feature measurements is also discussed. Computer simulated experiments in character recognition are shown to illustrate the feasibility of this approach.     

A new approach for stabilizing nonlinear systems with time delays

The proportional parallel distributed compensation (PPDC) approach is utilized to stabilize time‐delay systems modeled by Takagi‐Sugeno fuzzy models in this article. Based on the Lyapunov stability analysis, stability conditions concerning asymptotical stability of time‐delay systems are established. The main advantage of the PPDC approach over the parallel distributed compensation (PDC) approach is that fewer adjustable parameters are needed to ensure stability. Moreover, the procedure of finding common matrices P and S is simplified and the number of Lyapunov inequalities is reduced significantly. The entire PPDC design procedure employing linear matrix inequalities (LMIs) is presented. A numerical example of stabilizing a continuous stirred tank reactor (CSTR) is given to illustrate salient features of the new approach. © 2002 Wiley Periodicals, Inc.     

A dynamic programming approach to GA-based heuristic for multi-period CF problems

In this paper we have introduced a multi-period cell formation (CF) model which is more computationally challenging than the most comprehensive CF models in the literature. A dynamic programming (DP) based approach coupled with GA-based heuristic is proposed to solve the multi-period problem. Since, the introduced dynamic programming is general and can be applied to any GA-based heuristic with full rejuvenation cycles to solve the multi-period part of the model, we focused only on the DP approach in this paper but have explained the interface with the GA-based heuristic. Illustrative example has been provided that clarifies the application of DP-heuristic. The performance of the DP-heuristic has been evaluated against LINGO and multi period GA-based heuristic.     

A robust observer for discrete time nonlinear systems

This paper extends the results developed in (Ciccarella et al., 1993) and presents a robust observer for discrete time nonlinear systems. A simple, robust and easy to implement algorithm is given whose convergence properties are guaranteed for autonomous and forced systems. Combined parameter and state estimation is made for a numerical example, which compares the robust observer to the observer given in (Ciccarella et al., 1993).     

A quasi-newton differential dynamic programming algorithm for discrete-time optimal control

We develop a quasi-Newton differential dynamic programming algorithm (QDDP) for discrete-time optimal control problems. In the spirit of dynamic programming, the quasi-Newton approximations are performed in a stagewise manner. We establish the global convergence of the method and also show a superlinear convergence rate. Among other advantages of the QDDP method, second derivatives need not be calculated. In theory, the computational effort of each recursion grows proportionally to the number of stages N, whereas with conventional quasi-Newton techniques which do not take advantage of the optimal control problem structure, the growth is as N². Computational results are also reported.     

A new method for the control of discrete nonlinear dynamic systems using neural networks

A new controller design method for nonaffine nonlinear dynamic systems is presented in this paper. An identified neural network model of the nonlinear plant is used in the proposed method. The method is based on a new control law that is developed for any discrete deterministic time-invariant nonlinear dynamic system in a subregion Psi(x), of an asymptotically stable equilibrium point of the plant. The performance of the control law is not necessarily dependent on the distance between the current state of the plant and the equilibrium state if the nonlinear dynamic system satisfies some mild requirements in Psi(x). The control law is simple to implement and is based on a novel linearization of the input-output model of the plant at each instant in time. It can be used to control both minimum phase and nonminimum phase nonaffine nonlinear plants. Extensive empirical studies have confirmed that the control law can be used to control a relatively general class of highly nonlinear multiinput-multioutput (MIMO) plants.     

A new approach to static output feedback stabilization of linear dynamic systems

The topic of this study is output feedback control of linear control system with output. Use of condensed forms of the linear system under static output feedback (SOF) control is made to derive sufficient conditions for stabilization. A minimal-order control-free observer is presented.     

Convex dynamic programming for hybrid systems

A classical linear programming approach to optimization of flow or transportation in a discrete graph is extended to hybrid systems. The problem is finite dimensional if the state space is discrete and finite, but becomes infinite dimensional for a continuous or hybrid state space. It is shown how strict lower bounds on the optimal loss function can be computed by gridding the continuous state space and restricting the linear program to a finite-dimensional subspace. Upper bounds can be obtained by evaluation of the corresponding control laws.     

A new sliding function for discrete predictive sliding mode control of time delay systems

The control of time delay systems is still an open area for research. This paper proposes an enhanced model predictive discrete-time sliding mode control with a new sliding function for a linear system with state delay. Firstly, a new sliding function including a present value and a past value of the state, called dynamic surface, is designed by means of linear matrix inequalities （LMIs）. Then, using this dynamic function and the rolling optimization method in the predictive control strategy, a discrete predictive sliding mode controller is synthesized. This new strategy is proposed to eliminate the undesirable effect of the delay term in the closed loop system. Also, the designed control strategy is more robust, and has a chattering reduction property and a faster convergence of the system s state. Finally, a numerical example is given to illustrate the effectiveness of the proposed control.     

A Lyapunov approach to analysis of discrete singular systems

In this paper, a new type generalized Lyapunov equation for discrete singular systems is proposed. Then it is applied to study problems such as pole clustering, controllability and observability for discrete singular systems. First, some necessary and sufficient conditions for pole clustering are derived via the solution of this new type Lyapunov equation. Further, the relationship between the solution of the Lyapunov equation and structure properties of discrete singular systems will be investigated based on these results. Finally, a type of generalized Riccati equation is proposed and its solution is used to design state feedback law for discrete singular systems such that all the finite poles of the closed-loop systems are clustered into a specified disk.     

A mean field approach for optimization in discrete time

This paper investigates the limit behavior of Markov decision processes made of independent objects evolving in a common environment, when the number of objects (N) goes to infinity. In the finite horizon case, we show that when the number of objects becomes large, the optimal cost of the system converges to the optimal cost of a discrete time system that is deterministic. Convergence also holds for optimal policies. We further provide bounds on the speed of convergence by proving second order results that resemble central limits theorems for the cost and the state of the Markov decision process, with explicit formulas for the limit. These bounds (of order \(1/\sqrt{N}\)) are proven to be tight in a numerical example. One can even go further and get convergence of order \(\sqrt{\log N}/N\) to a stochastic system made of the mean field limit and a Gaussian term. Our framework is applied to a brokering problem in grid computing. Several simulations with growing numbers of processors are reported. They compare the performance of the optimal policy of the limit system used in the finite case with classical policies by measuring its asymptotic gain. Several extensions are also discussed. In particular, for infinite horizon cases with discounted costs, we show that first order limits hold and that second order results also hold as long as the discount factor is small enough. As for infinite horizon cases with non-discounted costs, examples show that even the first order limits may not hold.     

Dynamic programming for 2-D discrete linear systems

The authors calculate the optimal control of 2-D discrete linear systems using a dynamic programming method. It is assumed that the system is described with Roesser's state-space equations for which a 2-D sequence of inputs minimizing the given performance criterion is calculated. The method is particularly suitable for problems with bounded states and controls, although it can also be applied for unbounded cases. One numerical example is given     

Skeletal based programming for dynamic programming on MultiGPU systems

Current parallel systems composed of mixed multi/manycore systems and/with GPUs become more complex due to their heterogeneous nature. The programmability barrier inherent to parallel systems increases almost with each new architecture delivery. The development of libraries, languages, and tools that allow an easy and efficient use in this new scenario is mandatory. Among the proposals found to broach this problem, skeletal programming appeared as a natural alternative to easy the programmability of parallel systems in general, but also the GPU programming in particular. In this paper, we develop a programming skeleton for Dynamic Programming on MultiGPU systems. The skeleton, implemented in CUDA, allows the user to execute parallel codes for MultiGPU just by providing sequential C++ specifications of her problems. The performance and easy of use of this skeleton has been tested on several optimization problems. The experimental results obtained over a cluster of Nvidia Fermi prove the advantages of the approach.