Decentralised output feedback control of Markovian jump interconnected systems with unknown interconnections

The problem of decentralised output feedback control is addressed for Markovian jump interconnected systems with unknown interconnections and general transition rates (TRs) allowed to be unknown or known with uncertainties. A class of decentralised dynamic output feedback controllers are constructed, and a cyclic-small-gain condition is exploited to dispose the unknown interconnections so that the resultant closed-loop system is stochastically stable and satisfies an H_∞ performance. With slack matrices to cope with the nonlinearities incurred by unknown and uncertain TRs in control synthesis, a novel controller design condition is developed in linear matrix inequality formalism. Compared with the existing works, the proposed approach leads to less conservatism. Finally, two examples are used to illustrate the effectiveness of the new results.     

Analysis of discrete-time batch service renewal input queue with multiple working vacations

This paper investigates a discrete-time single server batch service queue with multiple working vacations wherein arrivals occur according to a discrete-time renewal process. The server works with a different service rate rather than completely stopping during the vacation period. The service is performed in batches and the server takes a vacation when the system does not have any waiting customers at a service completion epoch or a vacation completion epoch. We present a recursive method, using the supplementary variable technique to obtain the steady-state queue-length distributions at pre-arrival, arbitrary and outside observer’s observation epochs. The displacement operator method is used to solve simultaneous non-homogeneous difference equations. Some performance measures and waiting-time distribution in the system have also been discussed. Finally, numerical results showing the effect of model parameters on key performance measures are presented.     

Conditioned asymptotics for tail probabilities in large multiplexers

Consider a buffer whose input is a superposition of L independent identical sources, and which is served at rate sL. Recent work has shown that, under very general circumstances, the stationary tail probabilities for the queue of unfinished work Q in the buffer have the asymptotics P[Q > Lb] ≈ e^−LI(b) for large L. Here the shape function, I, is obtained from a variational expression involving the transient log cumulant generating function of the arrival process.

In this paper, we extend this analysis to cover time-dependent asymptotics for Markov arrival processes subject to conditioning at some instant. In applications we envisage that such conditioning would arise due to knowledge of the queue at a coarse-grained level, for example of the number of current active sources. We show how such partial knowledge can be used to predict future tail probabilities by use of a time dependent, conditioned shape function. We develop some heuristics to describe the time-dependent shape function in terms of a reduced set of quantities associated with the underlying arrivals process and show how to calculate them for renewal arrivals and a class of ON-OFF arrivals. This bypasses the full variational calculation of the shape function for such models.     

On a single server queue with two-stage heterogeneous service and deterministic server vacations

We analyse a single server queue with Poisson arrivals, two stages of heterogeneous service with different general (arbitrary) service time distributions and binomial schedule server vacations with deterministic (constant) vacation periods. After first-stage service the server must provide the second stage service. However, after the second stage service, he may take a vacation or may decide to stay on in the system. For convenience, we designate our model as M/G ₁, G ₂/D/1 queue. We obtain steady state probability generating function of the queue length for various states of the server. Results for some particular cases of interest such as M/E_{k 1} , E_{k 2} /D/1, M/M ₁, M ₂/D/1, M/E _k /D/1 and M/G ₁, G ₂/1 have been obtained from the main results and some known results including M/E_k /1 and M/G/1 have been derived as particular cases of our particular cases.     

A non-equilibrium analysis and control framework for active queue management

We present a non-equilibrium analysis and control approach for the Active Queue Management (AQM) problem in communication networks. Using simplified fluid models, we carry out a bifurcation study of the complex dynamic queue behavior to show that non-equilibrium methods are essential for analysis and optimization in the AQM problem. We investigate an ergodic theoretic framework for stochastic modeling of the non-equilibrium behavior in deterministic models and use it to identify parameters of a fluid model from packet level simulations. For computational tractability, we use set-oriented numerical methods to construct finite-dimensional Markov models, including control Markov chains and hidden Markov models. Subsequently, we develop and analyze an example AQM algorithm using a Markov Decision Process (MDP) based control framework. The control scheme developed is optimal with respect to a reward function, defined over the queue size and aggregate flow rate. We implement and simulate our illustrative AQM algorithm in the ns-2 network simulator. The results obtained confirm the theoretical analysis and exhibit promising performance when compared with well-known alternative schemes under persistent non-equilibrium queue behavior.     

Passivity analysis of stochastic delayed neural networks with Markovian switching

In this paper, the problem of passivity analysis is investigated for a class of stochastic delayed neural networks with Markovian switching. By applying Lyapunov functional and free-weighting matrix, delay-dependent/independent passivity criteria are presented in terms of linear matrix inequalities. The results herein include existing ones for neural networks without Markovian switching as special cases. An example is given to demonstrate the effectiveness of the proposed criteria.     

Learning and planning in environments with delayed feedback

This work considers the problems of learning and planning in Markovian environments with constant observation and reward delays. We provide a hardness result for the general planning problem and positive results for several special cases with deterministic or otherwise constrained dynamics. We present an algorithm, Model Based Simulation, for planning in such environments and use model-based reinforcement learning to extend this approach to the learning setting in both finite and continuous environments. Empirical comparisons show this algorithm holds significant advantages over others for decision making in delayed-observation environments.     

Stability analysis of multiserver discrete-time queueing systems with renewal-type server interruptions

For many queueing systems, the server is not continuously available. Service interruptions may result from repair times after server failures, planned maintenance periods or periods during which customers from other queues are being served. These service interruptions cause an overall performance degradation which is most striking when interruptions can start while a customer is being served and his service has to start all over after the interruption. This is the so-called preemptive repeat service discipline. This paper investigates stability conditions for discrete-time queueing systems with preemptive server interruptions. Under renewal assumptions for arrival, service and interruption processes, sufficient conditions for the positive recurrence of the single-server and multiserver queueing processes are established for the preemptive repeat different and the preemptive resume service disciplines.     

An MX/G/1 queue with randomized working vacations and at most J vacations

This paper treats a bulk arrival queue with randomized working vacation policy. Whenever the system becomes empty, the server takes a vacation. During the vacation period, customers are to be served at a lower rate. Once the vacation ends, the server will return to the normal working state and begin to serve the customers in the system if any. Otherwise, the server either remains idle with probability p or leaves for another vacation with probability 1?p. This pattern continues until the number of vacations taken reaches J. If the system is empty at the end of the Jth vacation, the server will wait idly for a new arrival. By using supplementary variable technique, we derive the system size distribution at arbitrary epoch, at departure epoch and at busy period initial epoch, as well as some important system characteristics. Numerical examples are provided to illustrate the influence of system parameters on several performance measures.     

Predicting departure times in multi-stage queueing systems

We develop an approximation model for the state-dependent sojourn time distribution of customers or orders in a multi-stage, multi-server queueing system, when interarrival and service times can take on general distributions. The model can be used to make probabilistic statements about the departure time of a customer or order, given the number and location of customers currently in process or waiting, and these probabilities can be recomputed while waiting at any point during the sojourn time. The model uses phase-type distributions and a new method to estimate the remaining processing times of customers in service when the sojourn time distribution is computed.     

Adaptive control of service in queueing systems

We consider the service-rate selection problem in an M/G/1 queueing system with unknown arrival rate and average cost criterion. An optimal adaptive policy is determined using recently developed results on parameter estimation and adaptive control of semi-Markov processes.     

Optimal resource allocation for time-reservation systems

This paper studies the optimal resource allocation in time-reservation systems. Customers arrive at a service facility and receive service in two steps; in the first step information is gathered from the customer, which is then sent to a pool of computing resources, and in the second step the information is processed after which the customer leaves the system. A central decision maker has to decide when to reserve computing power from the pool of resources, such that the customer does not have to wait for the start of the second service step and that the processing capacity is not wasted due to the customer still being serviced at the first step. The decision maker simultaneously has to decide on how many processors to allocate for the second processing step such that reservation and holding costs are minimized. Since an exact analysis of the system is difficult, we decompose the system into two parts which are solved sequentially leading to nearly optimal solutions. We show via dynamic programming that the near-optimal number of processors follows a step function with as an extreme policy the bang-bang control. Moreover, we provide new fundamental insights in the dependence of the near-optimal policy on the distribution of the information gathering times. Numerical experiments demonstrate that the near-optimal policy closely matches the performance of the optimal policy of the original problem.     

Lead time control in multi-class multi-stage assembly systems with finite capacity

In this paper, we model multi-class multi-stage assembly systems with finite capacity as queueing networks. It is assumed that different classes (types) of products are produced by the production system and products’ orders for different classes are received according to independent Poisson processes. Each service station of the queueing network specifies a manufacturing or assembly operation, in that processing times for different types of products are independent and exponentially distributed random variables with service rates, which are controllable, and the queueing discipline is First Come First Served (FCFS). Different types of products may be different in their routing sequences of manufacturing and assembly operations. For modeling multi-class multi-stage assembly systems, we first consider every class separately and convert the queueing network of each class into an appropriate stochastic network. Then, by using the concept of continuous-time Markov processes, a system of differential equations is created to obtain the distribution function of manufacturing lead time for any type of product, which is actually the time between receiving the order and the delivery of finished product. Furthermore, we develop a multi-objective model with three conflicting objectives to optimally control the service rates, and use goal attainment method to solve a discrete-time approximation of the original multi-objective continuous-time problem.     

Markovian jump delayed Hopfield networks with multiplicative noise

A non-symmetric version of Hopfield networks subject to state-multiplicative noise, pure time delay and Markov jumps is considered. Such networks arise in the context of visuo-motor control loops and may, therefore, be used to mimic their complex behavior. In this paper, we adopt the Lur’e-Postnikov systems approach to analyze the stochastic stability and the L₂ gain of generalized Hopfield networks including these effects.     

Optimal maintenance policy for multi-component systems under Markovian environment changes

In this paper, we study multi-component systems, which environmental conditions and opportunistic maintenance (OM) involve. Environmental conditions will exert an influence on deterioration processes of the components in the system. For each component, the worse the environmental conditions are, the faster its deterioration speed is. We want to determine when to preventively maintain each component under such environmental influence. Our purpose is to minimize its long-run average maintenance cost. We decompose such a multi-component system into mutually influential single-component systems, and formulate the maintenance problem of each component as a Markov decision process (MDP). Under some reasonable assumptions, we prove the existence of the optimal (n_r, N_r) type policy for each component. A policy iteration method is used to calculate its optimal maintenance policy. Based on the method, we develop an iterative approximation algorithm to obtain an acceptable maintenance policy for a multi-component system. Numerical examples find that environmental conditions and OM pose significant effects on a maintenance policy.     

On the overflow processes from the PH1 + PH2/M/S/K queue with two independent PH-renewal inputs

This paper presents an analysis of overflow processes from a PH₁ + PH₂/M/S/K queue having two independent phase type renewal input streams. Both the superposed overflow process and individual overflow processes for the PH₁- and PH₂-streams are analyzed using first passage time distributions for the number of customers in the system. Each overflow process is characterized as a Markov renewal process. The nth moment of the number of customers in an infinite server group to which these overflows have been offered is derived using a theory for the MR/M/∞ queue with a Markov renewal input. The numerical examples for means and variance-to-mean ratios (peakednesses) of the individual overflow streams are given for an H₂ + H₂/M/S/S queue with interrupted Poisson inputs, which is a vital model for telephone network planning. In addition, overflow traffic characteristics are discussed by using these examples.     

A tandem queue with general bulk service and servers' “vacation”

This paper analyses two queueing models consisting of two units I and II connected in series, separated by a finite buffer of size N. In both models, unit I has only one exponential server capable of serving customers one at a time and unit II consist of c parallel exponential servers, each of them serving customers in groups according to general bulk service rules. When the queue length in front of unit II is less than the minimum of batch size, the free servers take a vacation. On return from vacation, if the queue length is less than the minimum, they leave for another vacation in the first model, whereas in the second model they wait in the system until they get the minimum number of customers and then start servicing. The steady-state probability vector of the number of customers waiting and receiving service in unit I and waiting in the buffer is obtained for both the models, using the modified matrix geometric method. Numerical results are also presented.     

Normalisation design for delayed singular Markovian jump systems based on system transformation technique

This paper deals with the problem of normalisation and stabilisation for time-varying delayed singular Markovian jump systems by virtue of proportional-derivative state feedback control. Based on the system transformation technique, the normalisation problem of singular systems is changed into the stabilisation problem of the augmented singular systems. Sufficient conditions are provided in terms of linear matrix inequalities by constructing mode-dependent and delay-dependent Lyapunov–Krasovskii functional. Simulation examples including an oil catalytic cracking process are utilised to verify the effectiveness and usefulness of the proposed method.     

Probabilistic systems coalgebraically: A survey

We survey the work on both discrete and continuous-space probabilistic systems as coalgebras, starting with how probabilistic systems are modeled as coalgebras and followed by a discussion of their bisimilarity and behavioral equivalence, mentioning results that follow from the coalgebraic treatment of probabilistic systems. It is interesting to note that, for different reasons, for both discrete and continuous probabilistic systems it may be more convenient to work with behavioral equivalence than with bisimilarity.     

Analysis of a discrete-time queue with general three-state Markovian traffic sources

In this paper, we analyze the performance characteristics of an ATM switch, where cell arrivals are generated according to a three-state Markovian arrival law. This arrival process is an extension for the two-state ON/OFF source model that has already been extensively treated in the literature, and it allows a much more flexible modeling of a wide variety of VBR sources. As a result, for the finite storage capacity case, the system contents distribution, as well as the cell loss ratio are derived. On the other hand, for the infinite storage capacity case, a method for deriving an expression for the probability generating function of the system contents is presented. Furthermore, an approximation (consisting of multiple geometric terms) for the tail distribution of the system contents is given. The accuracy of this approximation as well as the impact of the traffic parameters on the buffer behavior are discussed via some numerical examples.