Multi-objective reliability optimization for dissimilar-unit cold-standby systems using a genetic algorithm

A genetic algorithm approach is used to solve a multi-objective discrete reliability optimization problem in a k dissimilar-unit non-repairable cold-standby redundant system. Each unit is composed of a number of independent components with generalized Erlang distributions arranged in a series–parallel configuration. There are multiple component choices with different distribution parameters available for being replaced with each component of the system. The objective of the reliability optimization problem is to select the best components, from the set of available components, to be placed in the standby system in order to minimize the initial purchase cost of the system, maximize the system MTTF (mean time to failure), minimize the system VTTF (variance of time to failure) and also maximize the system reliability at the mission time. Finally, we apply a genetic algorithm with double strings using continuous relaxation based on reference solution updating (GADSCRRSU) to solve this multi-objective problem, using goal attainment formulation. The results are also compared against the results of a discrete-time approximation technique to show the efficiency of the proposed GA approach.     

Predicting departure times in multi-stage queueing systems

We develop an approximation model for the state-dependent sojourn time distribution of customers or orders in a multi-stage, multi-server queueing system, when interarrival and service times can take on general distributions. The model can be used to make probabilistic statements about the departure time of a customer or order, given the number and location of customers currently in process or waiting, and these probabilities can be recomputed while waiting at any point during the sojourn time. The model uses phase-type distributions and a new method to estimate the remaining processing times of customers in service when the sojourn time distribution is computed.     

Quadratic control of stochastic hybrid systems with renewal transitions

We study the quadratic control of a class of stochastic hybrid systems with linear continuous dynamics for which the lengths of time that the system stays in each mode are independent random variables with given probability distribution functions. We derive a condition for finding the optimal feedback policy that minimizes a discounted infinite horizon cost. We show that the optimal cost is the solution to a set of differential equations with unknown boundary conditions. Furthermore, we provide a recursive algorithm for computing the optimal cost and the optimal feedback policy. The applicability of our result is illustrated through a numerical example, motivated by stochastic gene regulation in biology.     

Decentralised output feedback control of Markovian jump interconnected systems with unknown interconnections

The problem of decentralised output feedback control is addressed for Markovian jump interconnected systems with unknown interconnections and general transition rates (TRs) allowed to be unknown or known with uncertainties. A class of decentralised dynamic output feedback controllers are constructed, and a cyclic-small-gain condition is exploited to dispose the unknown interconnections so that the resultant closed-loop system is stochastically stable and satisfies an H_∞ performance. With slack matrices to cope with the nonlinearities incurred by unknown and uncertain TRs in control synthesis, a novel controller design condition is developed in linear matrix inequality formalism. Compared with the existing works, the proposed approach leads to less conservatism. Finally, two examples are used to illustrate the effectiveness of the new results.     

Probabilistic systems coalgebraically: A survey

We survey the work on both discrete and continuous-space probabilistic systems as coalgebras, starting with how probabilistic systems are modeled as coalgebras and followed by a discussion of their bisimilarity and behavioral equivalence, mentioning results that follow from the coalgebraic treatment of probabilistic systems. It is interesting to note that, for different reasons, for both discrete and continuous probabilistic systems it may be more convenient to work with behavioral equivalence than with bisimilarity.     

Genetic algorithm for supply planning in two-level assembly systems with random lead times

This paper examines supply planning for two-level assembly systems under lead time uncertainties. It is supposed that the demand for the finished product and its due date are known. The assembly process at each level begins when all necessary components are in inventory. If the demand for the finished product is not delivered at the due date, a tardiness cost is incurred. In the same manner, a holding cost at each level appears if some components needed to assemble the same semi-finished product arrive before beginning the assembly at this level. It is assumed also that the lead time at each level is a random discrete variable. The expected cost is composed of the tardiness cost for finished product and the holding costs of components at levels 1 and 2. The objective is to find the release dates for the components at level 2 in order to minimize the total expected cost. For this new problem, a genetic algorithm is suggested. The proposed algorithm is evaluated with a variety of supply chain settings in order to verify its robustness across different supply chain scenarios. Moreover, the effect of a local search on the performance of the Genetic Algorithm in terms of solution quality, convergence and computation time is also investigated.     

Optimal control of markov chains admitting strong and weak interactions

This paper is devoted to the study of an optimal control problem for a Markov chain with generator B + εA, where ε is a small parameter. It is shown that an approximate solution can be calculated by a policy improvement algorithm involving computations relative to an ‘aggregated’ problem (the dimension of which is given by N, the number of ergodic sets for the B matrix) together with a family of ‘decentralized’ problems (the dimensions of which are given by the number of elements in each ergodic set for the B matrix).     

Solving two-machine assembly scheduling problems with inventory constraints

This paper considers a scheduling problem with component availability constraints in a supply chain consisting of two manufacturing facilities and a merge-in-transit facility. Three mixed-integer programming (MIP) models and a constraint programming (CP) model are compared in an extensive numerical study. Results show that when there are no components shared among the two manufacturers, the MIP model based on time-index variables is the best for proving optimality for problems with short processing times whereas the CP model tends to perform better than the others for problems with a large range of processing times. When shared components are added, the performance of all models deteriorates, with the time-indexed MIP providing the best results. By explicitly modelling the dependence of scheduling decisions on the availability of component parts, we contribute to the literature on the integration of inventory and scheduling decisions, which is necessary for solving realistic supply chain problems.     

Control of manufacturing systems with a two-value, production-dependent failure rate

In this paper we consider a failure prone, single machine, single part-type, limited inventory, manufacturing system subject to a non-homogeneous Markov failure/repair process with the failure rate depending on the production rate through a two-value function. The problem is to minimize a cost function which includes a penalty for waiting customers. We derive and prove the optimality of a policy which depends on a threshold but is not the so-called hedging point policy, optimal for the Markov homogeneous case.     

The control of a two-level Markov decision process by time aggregation

The solution of Markov Decision Processes (MDPs) often relies on special properties of the processes. For two-level MDPs, the difference in the rates of state changes of the upper and lower levels has led to limiting or approximate solutions of such problems. In this paper, we solve a two-level MDP without making any assumption on the rates of state changes of the two levels. We first show that such a two-level MDP is a non-standard one where the optimal actions of different states can be related to each other. Then we give assumptions (conditions) under which such a specially constrained MDP can be solved by policy iteration. We further show that the computational effort can be reduced by decomposing the MDP. A two-level MDP with M upper-level states can be decomposed into one MDP for the upper level and M to M(M-1) MDPs for the lower level, depending on the structure of the two-level MDP. The upper-level MDP is solved by time aggregation, a technique introduced in a recent paper [Cao, X.-R., Ren, Z. Y., Bhatnagar, S., Fu, M., & Marcus, S. (2002). A time aggregation approach to Markov decision processes. Automatica, 38(6), 929-943.], and the lower-level MDPs are solved by embedded Markov chains.     

Optimal control for multi-stage and continuous-time linear singular systems

In this paper, optimal control problems for multi-stage and continuous-time linear singular systems are both considered. The singular systems are assumed to be regular and impulse-free. First, a recurrence equation is derived according to Bellman's principle of optimality in dynamic programming. Then, by applying the recurrence equation, bang-bang optimal controls for the control problems with linear objective functions subject to two types of multi-stage singular systems are obtained. Second, employing the principle of optimality, a equation of optimality for settling the optimal control problem subject to a class of continuous-time singular systems is proposed. The optimal control problem may become simpler through solving this equation of optimality. Two numerical examples and a dynamic input–output model are presented to show the effectiveness of the results obtained.     

Shaping multi-agent systems with gradient reinforcement learning

An original reinforcement learning (RL) methodology is proposed for the design of multi-agent systems. In the realistic setting of situated agents with local perception, the task of automatically building a coordinated system is of crucial importance. To that end, we design simple reactive agents in a decentralized way as independent learners. But to cope with the difficulties inherent to RL used in that framework, we have developed an incremental learning algorithm where agents face a sequence of progressively more complex tasks. We illustrate this general framework by computer experiments where agents have to coordinate to reach a global goal. This work has been conducted in part in NICTA’s Canberra laboratory.     

Numerical analysis of continuous time Markov decision processes over finite horizons

Continuous time Markov decision processes (CTMDPs) with a finite state and action space have been considered for a long time. It is known that under fairly general conditions the reward gained over a finite horizon can be maximized by a so-called piecewise constant policy which changes only finitely often in a finite interval. Although this result is available for more than 30 years, numerical analysis approaches to compute the optimal policy and reward are restricted to discretization methods which are known to converge to the true solution if the discretization step goes to zero. In this paper, we present a new method that is based on uniformization of the CTMDP and allows one to compute an ε-optimal$\epsilon \text{-optimal}$ policy up to a predefined precision in a numerically stable way using adaptive time steps.     

Input-output finite time stabilization of linear systems

Bounded Input Bounded Output (BIBO) stability is usually studied when only the input-output behavior of a dynamical system is of concern. The present paper investigates the analogous concept in the framework of Finite Time Stability (FTS), namely the Input-Output FTS (IO-FTS). FTS has been already investigated in several papers in terms of state boundedness, whereas in this work we deal with the characterization of the input-output behavior. Sufficient conditions are given, concerning the class of L₂ and L_∞ input signals, for the analysis of IO-FTS and for the design of a static state feedback controller, guaranteeing IO-FTS of the closed-loop system. The effectiveness of the proposed results is eventually illustrated by means of some numerical examples.     

Nonstationary discrete-time deterministic and stochastic control systems with infinite horizon

This article is about nonstationary nonlinear discrete-time deterministic and stochastic control systems with Borel state and control spaces, possibly noncompact control constraint sets, and unbounded costs. The control problem is to minimise an infinite-horizon total cost performance index. Using dynamic programming arguments we show that, under suitable assumptions, the optimal cost functions satisfy optimality equations, which in turn give a procedure to find optimal control policies.     

Repetitive control of servo systems with time delays

In many industrial robotic servo applications there is a need to track periodic reference signals and/or reject periodic disturbances. Moreover, time-delays are usually unavoidable in control systems due to the sensoring and communication delays. This paper presents an alternative repetitive control design for systems with constant time-delays in both forward and feedback control channels, which are dedicated to track/reject periodic signals. An additional delay is introduced together with the plant delays to construct an internal model for periodic signals, and a simple compensator based on the plant model inverse is utilized to stabilize the closed-loop system. Sufficient stability conditions of the closed-loop system and the robustness analysis against modeling uncertainties are studied. The proposed idea is further extended for general time-delay systems with only a delay term in the forward control channel. The “plug-in” structure used in conventional repetitive control designs is avoided, so that it leads to a simpler control configuration, i.e. only a proportional parameter and the cutoff frequency of a low-pass filter are required to be selected. Simulations based on a hard disk drive system and practical experiments on a rotary robotic servo system are provided to evaluate the effectiveness of the proposed method.     

Stability of digital control systems implemented in error-recoverable computers

Safety-critical, real-time applications make use of fault-tolerant digital control systems to achieve their performance goals. Even though these digital control systems are fault-tolerant, they are susceptible to errors induced by common-mode faults, since these errors cannot be masked by standard redundancy provisions. Such errors can be handled by special error correction mechanisms, which could require stopping the control law computations while the errors are removed. Thus, the effect of these special error recovery mechanisms on digital control systems needs to be understood. This paper presents a comprehensive study of the effect of three error recovery mechanisms–rollback, reset and cold restart–on the stability of digital closed-loop control systems. The effect of the faults, detected and handled by these error recovery mechanisms, on the digital control system is modelled by a set of interference maps. It is assumed that the arrival and departure of common-mode faults can be represented by a Markov process. The overall system behaviour is then described by a Markov jump linear model, whose stability is explored using standard techniques from the literature. The result of this analysis is a new metric, the minimum average interarrival spacing (MAIS), which is useful for comparing the performance of different error recovery mechanisms and for designing new fault-tolerant controllers. The theoretical results are illustrated via Monte Carlo simulations that show the effects of common-mode faults on the closed-loop stability of the longitudinal dynamics of an AFTI/F-16 aircraft.     

The risk element transmission theory research of multi-objective risk-time-cost trade-off

Risk management project is an important aspect of general project risk element transmission theory. To solve the multi-objective time-cost trade-off problem considering the risk elements effectively, this paper establishes an analytical model for multi-objective risk-time-cost trade-off problem based on general project risk element transmission theory. We divide risk elements into discrete model and continuous model to be discussed separately, and the two models for multi-objective risk-time-cost trade-off problem are established by taking Markov dynamic PERT network into classical PERT network. Thus, we combine Radial Basis Function (RBF) neural network to solve the discrete model of the problem. Finally, a practical example illustrates the effectiveness of the algorithm.     

Online pricing for bandwidth provisioning in multi-class networks

We consider the problem of pricing for bandwidth provisioning over a single link, where users arrive according to a known stochastic traffic model. The network administrator controls the resource allocation by setting a price at every epoch, and each user’s response to the price is governed by a demand function. We formulate this problem as a partially observable Markov decision process (POMDP), and explore two novel pricing schemes––reactive pricing and spot pricing––and compare their performance to appropriately tuned flat pricing. We use a gradient-ascent approach in all the three pricing schemes. We provide methods for computing unbiased estimates of the gradient in an online (incremental) fashion. Our simulation results show that our novel schemes take advantage of the known underlying traffic model and significantly outperform the model-free pricing scheme of flat pricing.