Reward model solution methods with impulse and rate rewards: an algorithm and numerical results

Reward models have become an important method for specifying performability models for many types of systems. Many methods have been proposed for solving reward models, but no method has proven itself to be applicable over all system classes and sizes. Furthermore, specification of reward models has usually been done at the state level, which can be extremely cumbersome for realistic models. We describe a method to specify reward models as stochastic activity networks (SANs) with impulse and rate rewards, and a method by which to solve these models via uniformization. The method is an extension of one proposed by de Souza e Silva and Gail in which impulse and rate rewards are specified at the SAN level, and solved in a single model. Furthermore, we propose a new technique for discarding paths in the uniformized process whose contribution to the reward variable is minimal, which greatly reduces the time and space required for a solution. A bound is calculated on the error introduced by this discarding, and its effectiveness is illustrated through the study of the performability and availability of a degradable multi-processor system.     

On Markov reward modelling with FSPNs

In this paper, fluid stochastic Petri nets (FSPNs) will be used for modelling reward in a performability model. Two variations of a known performability model are presented in order to demonstrate the ability of FSPNs in modelling accumulated rate reward as well as accumulated impulse reward. In the first model two fluid places are used, one of which represents the profit (reward) obtained by operating the system and the other one the buffer that is approximated continuously. In the second model only one fluid place is used, representing the costs (negative reward) arising due to repair of system components. The costs increase continuously at deterministic rate while the system is in state of repair (which is a rate reward in the model). Additional costs incur each time the buffer fails (which is an impulse reward in the model). With a numerical solution algorithm the distribution of the reward and its mean are computed. The accuracy of the numerical algorithm is studied by showing for the first model the impact of the choice of the discretization stepsizes on the obtained solution. Different boundary conditions are discussed for the second model.     

Specification techniques for Markov reward models

Markov reward models (MRMs) are commonly used for the performance, dependability, and performability analysis of computer and communication systems. Many papers have addressed solution techniques for MRMs. Far less attention has been paid to the specification of MRMs and the subsequent derivation of the underlying MRM. In this paper we only briefly address the mathematical aspects of MRMs. Instead, emphasis is put on specification techniques. In an application independent way, we distinguish seven classes of specification techniques: stochastic Petri nets, queuing networks, fault trees, production rule systems, communicating processes, specialized languages, and hybrid techniques. For these seven classes, we discuss the main principles, give examples and discuss software tools that support the use of these techniques. An overview like this has not been presented in the literature before. Finally, the paper addresses the generation of the underlying MRM from the high-level specification, and indicates important future research areas. This work was supported in part by the Naval Surface Warfare Center under contract N60921-92-C-0161 and by the National Science Foundation under grant CCR-9108114.     

计算机系统与计算机网络中的动态优化:模型、求解与应用

动态优化是计算机系统与计算机网络中进行资源分配与任务调度等方面研究所采用的主要理论工具之一.目前,国内外已开展大量研究,致力于深化动态优化的理论研究与工程应用.文中从模型、求解与应用3个角度,对马尔可夫决策过程动态优化理论模型进行了综述,并重点介绍了将动态优化理论与随机Petri网理论相结合的马尔可夫决策Petri网和随机博弈网模型,详细讨论了这些模型的建模方法、求解算法与一些应用实例.最后,对全文进行了总结,并对未来可能的研究方向进行了展望.     

Lumping partially symmetrical stochastic models

The performance and dependability evaluation of complex systems by means of dynamic stochastic models (e.g. Markov chains) may be impaired by the combinatorial explosion of their state space. Among the possible methods to cope with this problem, symmetry-based ones can be applied to systems including several similar components. Often however these systems are only partially symmetric: their behavior is in general symmetric except for some local situation when the similar components need to be differentiated.In this paper two methods to efficiently analyze partially symmetrical models are presented in a general setting and the requirements for their efficient implementation are discussed. Some case studies are presented to show the methods’ effectiveness and their applicative interest.     

Queueing Petri nets with product form solution

The product form results recently published for stochastic Petri nets are combined with the well-known product form results for queueing networks in the model formalism of queueing Petri nets yielding the class of product form queueing Petri nets. This model class includes stochastic Petri nets with product form solution and BCMP queueing networks as special cases. We introduce an arrival theorem for the model class and present an exact aggregation approach extending known approaches from queueing networks.     

Markov regenerative stochastic Petri nets with general execution policies: supplementary variable analysis and a prototype tool

Stochastic Petri nets (SPNs) with general firing time distributions are considered. Generally timed transitions can have general execution policies: the preemption policy may be preemptive repeat different (prd) or preemptive resume (prs) and the firing time distribution can be marking-dependent. A stationary analysis method covering all possible combinations is presented by means of supplementary variables. The method is implemented in a prototype tool SPNica which is based on Mathematica. The use of the general execution policies is illustrated by a WWW server model.     

A congruence relation for sPBC

In this paper we define a congruence relation for regular terms of sPBC (stochastic Petri Box Calculus), by means of which we identify those processes that have the same behaviour, not only in terms of the multiactions that they can perform, but also taking into account the stochastic information that they have associated. In order to define this equivalence relation we have to define an adequate semantics for the synchronization operator, as well as a new labelled transition system for regular terms of sPBC.     

Prediction of business process durations using non-Markovian stochastic Petri nets

Companies need to efficiently manage their business processes to deliver products and services in time. Therefore, they monitor the progress of individual cases to be able to timely detect undesired deviations and to react accordingly. For example, companies can decide to speed up process execution by raising alerts or by using additional resources, which increases the chance that a certain deadline or service level agreement can be met. Central to such process control is accurate prediction of the remaining time of a case and the estimation of the risk of missing a deadline.To achieve this goal, we use a specific kind of stochastic Petri nets that can capture arbitrary duration distributions. Thereby, we are able to achieve higher prediction accuracy than related approaches. Further, we evaluate the approach in comparison to state of the art approaches and show the potential of exploiting a so far untapped source of information: the elapsed time since the last observed event. Real-world case studies in the financial and logistics domain serve to illustrate and evaluate the approach presented.     

基于Petri网的批处理调度研究

批处理过程存在于复杂的动态环境中,来自主客观的干扰及问题固有的易变性,会导致各种过程参数的变化,因此,需要研究对意外事件作出快速反应的动态调度方法,以捕捉生产环境的实时变化。该文针对批处理过程中最常出现的操作处理时间波动,提出了基于Petri网仿真技术的批处理过程动态调度方法。仿真结果表明,该方法能有效地改善调度性能,为批处理过程动态调度的研究提供了新思路。     

Iterative analysis of Markov regenerative models

Conventional algorithms for the steady-state analysis of Markov regenerative models suffer from high computational costs which are caused by densely populated matrices. In this paper, a new algorithm is suggested which avoids computing these matrices explicitly. Instead, a two-stage iteration scheme is used. An extended version of uniformization is applied as a subalgorithm to compute the required transient quantities “on-the fly”. The algorithm is formulated in terms of stochastic Petri nets. A detailed example illustrates the proposed concepts.     

A new LP formulation of the admission control problem modelled as an MDP under average reward criterion

The admission control problem can be modelled as a Markov decision process (MDP) under the average cost criterion and formulated as a linear programming (LP) problem. The LP formulation is attractive in the present and future communication networks, which support an increasing number of classes of service, since it can be used to explicitly control class-level requirements, such as class blocking probabilities. On the other hand, the LP formulation suffers from scalability problems as the number C of classes increases. This article proposes a new LP formulation, which, even if it does not introduce any approximation, is much more scalable: the problem size reduction with respect to the standard LP formulation is O((C?+?1)²/2 ^C ). Theoretical and numerical simulation results prove the effectiveness of the proposed approach.     

Planning without state space explosion: Petri net to Markov decision process

This paper explores the construction of a Markov decision process (MDP) from a predicate transition net representation of a planning problem, which can be optimised using dynamic programming methods. The construction methodology alleviates the problem of state space explosion.     

Availability modeling and analysis of a data center for disaster tolerance

Availability assessment of a data center with disaster tolerance (DT) is demanding for cloud computing based businesses. Previous work attempted to model and analyze the computing systems without a good consideration on disaster occurrence, unexpected failure of network connection and proper dependencies between subsystems in a data center. This paper presents a comprehensive availability model of a data center for DT using stochastic reward nets (SRN). The model incorporates (i) a typical two-level high availability (HA) configuration (i.e., active/active between sites and active/passive within a site), (ii) various fault and disaster tolerant techniques; (iii) dependencies between subsystems (e.g. between a host and virtual machines (VMs), between a network area storage (NAS) and VMs) and dependency between disastrous events and physical subsystems; and (iv) unexpected failures during data transmission between data centers. The constructed SRN model is analyzed on the basis of steady state analysis, downtime cost analysis, and sensitivity analysis with regard to major impacting parameters. The analysis results show the availability improvement of the disaster tolerant data center (DTDC) and featured system responses corresponding to the selected variables. The modeling and analysis of the DTDC in this paper provide a selection basis of designing for disasters in consideration of the trade-off between system availability and downtime cost with infrastructure construction cost.     

Tight bounds for the cover time of multiple random walks

We study the cover time of multiple random walks on undirected graphs G=(V,E). We consider k parallel, independent random walks that start from the same vertex. The speed-up is defined as the ratio of the cover time of a single random walk to the cover time of these k random walks. Recently, Alon et al. (2008) [5] derived several upper bounds on the cover time, which imply a speed-up of Ω(k) for several graphs; however, for many of them, k has to be bounded by O(logn). They also conjectured that, for any 1?k?n, the speed-up is at most O(k) on any graph. We prove the following main results:

•

We present a new lower bound on the speed-up that depends on the mixing time. It gives a speed-up of Ω(k) on many graphs, even if k is as large as n.

•

We prove that the speed-up is O(klogn) on any graph. For a large class of graphs we can also improve this bound to O(k), matching the conjecture of Alon et al.

•

We determine the order of the speed-up for any value of 1?k?n on hypercubes, random graphs and degree restricted expanders. For d-dimensional tori with d>2, our bounds are tight up to logarithmic factors.

•

Our findings also reveal a surprisingly sharp threshold behaviour for certain graphs, e.g., the d-dimensional torus with d>2 and hypercubes: there is a value T such that the speed-up is approximately min{T,k} for any 1?k?n.

     

Performance modeling of IEEE 802.11 wireless LANs with stochastic Petri nets

In 1997, IEEE standardized the physical layers and the medium access for wireless local area networks. This paper presents a performance study of the distributed coordination function, the fundamental contention-based access mechanism. Most performance studies adopt unchecked simplifying assumptions or do not reveal all details of the simulation model. We develop a stochastic Petri net model, which captures all relevant system aspects in a concise way. Simulation allows to quantify the influence of many mandatory features of the standard on performance, especially the backoff procedure, extended interframe spaces, and the timing synchronization function. We identify conditions when simplifying assumptions commonly used in analytical modeling are justified. Applying these conditions, we derive a more compact and analytically tractable model from the detailed model.     

无线传感器网络中一种能量高效的分布式分簇算法

提出了一种适用于无线传感器网络的能量高效的分布式分簇算法（ＥＥＤＣ）,该算法使具有较高剩余能量及距离基站较近的节点有更大的机会成为簇头．理论分析表明该算法通信开销较小,而且有效地均衡了节点的能量消耗．为了确保ＥＥＤＣ 的正确性、完整性和可靠性,利用形式化方法———着色网对其关键属性进行建模和分析．仿真结果表明,ＥＥＤＣ 有效地延长了网络生命周期,提高了网络的能耗效率．     

Dependability evaluation of an air traffic control computing system

As air traffic over France is growing rapidly, the existing air traffic control (ATC) system has to evolve to satisfy the increasing demand. The selection of the new automated computing system (denoted CAUTRA) is based, among other things, on dependability evaluation. This paper is devoted to the dependability evaluation of the CAUTRA, however, emphasis is put on a subset: the regional control center (RCC). Starting from the analysis of the impact of CAUTRA failures on air traffic safety, five levels of service degradation are defined for the global system grading the effects of these failures on the service delivered to the controllers to ensure traffic safety. The RCC failure modes leading to these degradation levels are then defined and evaluated using stochastic Petri nets. The modeling approach consists in modeling the system as a set of modules interconnected via coupling mechanisms. The system model is constructed in several steps according to an incremental approach. Each step integrates the failure and recovery assumptions of an additional component and updates the model of the previous step by accounting for the impact of the new component on the behavior of those already included in the model. The application of this approach to the CAUTRA allowed us to analyze several configurations of the CAUTRA architecture and to identify improvement areas to minimize the impact of CAUTRA failures on air traffic safety.     

计算机网络的建模与分析

计算机网络具有既随机又相关的特点，因此选择计算机网络性能评价的数学工具要考虑到网络数据流的特性，排队模型是一种确定性的数学模型，用确定性的模型描述网络既随机又相关的动态行为是不够的。Ｐｅ＃簦颍? Ｎｅｔｓ（ＰＮ）能够比较方便地刻画网络的相关录枋鐾绲木赫⑴鲎埠妥枞樱校谓肜┏浞⒄苟吹乃婊校危梢员冉戏奖愕目袒缡录乃婊裕呓岷掀鹄创酉嗟背潭壬匣汉土思扑慊缃Ｓ敕治鲋涞     

Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains

We consider average reward Markov decision processes with discrete time parameter and denumerable state space. We are concerned with the following problem: Find necessary and sufficient conditions so that, for arbitrary bounded reward function, the corresponding average reward optimality equation has a bounded solution. This problem is solved for a class of systems including the case in which, under the action of any stationary policy, the state space is an irreducible positive recurrent class.