Parametric probabilistic transition systems for system design and analysis

We develop a model of parametric probabilistic transition Systems (PPTSs), where probabilities associated with transitions may be parameters. We show how to find instances of the parameters that satisfy a given property and instances that either maximize or minimize the probability of reaching a certain state. As an application, we model a probabilistic non-repudiation protocol with a PPTS. The theory we develop allows us to find instances that maximize the probability that the protocol ends in a fair state (no participant has an advantage over the others). A preliminary version of this paper was presented at SEFM’04 [LMT04]. 05 April 2006     

CSL model checking algorithms for QBDs

We present an in-depth treatment of model checking algorithms for a class of infinite-state continuous-time Markov chains known as quasi-birth death processes. The model class is described in detail, as well as the logic CSL to express properties of interest. Using a new property-independency concept, we provide model checking algorithms for all the CSL operators. Special emphasis is given to the time-bounded until operator for which we present a new and efficient computational procedure named uniformization with representatives. By the use of an application-driven dynamic stopping criterion, the algorithm stops whenever the property to be checked can be certified (or falsified). A comprehensive case study of a connection management system shows the versatility of our new algorithms.     

A simple graph theoretic characterization of reachability for positive linear systems

In this paper we consider discrete-time linear positive systems, that is systems defined by a pair (A,B) of non-negative matrices. We study the reachability of such systems which in this case amounts to the freedom of steering the state in the positive orthant by using non-negative control sequences. This problem was solved recently [Canonical forms for positive discrete-time linear control systems, Linear Algebra Appl., 310 (2000) 49]. However we derive here necessary and sufficient conditions for reachability in a simpler and more compact form. These conditions are expressed in terms of particular paths in the graph which is naturally associated with the system.     

Efficient processing of label-constraint reachability queries in large graphs

In this paper, we study a variant of reachability queries, called label-constraint reachability (LCR) queries. Specifically, given a label set S and two vertices u₁ and u₂ in a large directed graph G, we check the existence of a directed path from u₁ to u₂, where edge labels along the path are a subset of S. We propose the path-label transitive closure method to answer LCR queries. Specifically, we t4ransform an edge-labeled directed graph into an augmented DAG by replacing the maximal strongly connected components as bipartite graphs. We also propose a Dijkstra-like algorithm to compute path-label transitive closure by re-defining the “distance” of a path. Comparing with the existing solutions, we prove that our method is optimal in terms of the search space. Furthermore, we propose a simple yet effective partition-based framework (local path-label transitive closure+online traversal) to answer LCR queries in large graphs. We prove that finding the optimal graph partition to minimize query processing cost is a NP-hard problem. Therefore, we propose a sampling-based solution to find the sub-optimal partition. Moreover, we address the index maintenance issues to answer LCR queries over the dynamic graphs. Extensive experiments confirm the superiority of our method.     

Probabilistic systems coalgebraically: A survey

We survey the work on both discrete and continuous-space probabilistic systems as coalgebras, starting with how probabilistic systems are modeled as coalgebras and followed by a discussion of their bisimilarity and behavioral equivalence, mentioning results that follow from the coalgebraic treatment of probabilistic systems. It is interesting to note that, for different reasons, for both discrete and continuous probabilistic systems it may be more convenient to work with behavioral equivalence than with bisimilarity.     

On the reachability in any fixed time for positive continuous-time linear systems

This paper deals with the reachability of continuous-time linear positive systems. The reachability of such systems, which we will call here the strong reachability, amounts to the possibility of steering the state in any fixed time to any point of the positive orthant by using nonnegative control functions. The main result of this paper essentially says that the only strongly reachable positive systems are those made of decoupled scalar subsystems. Moreover, the strongly reachable set is also characterized.     

A new blocking problem from Java-based schedulers

We consider a form of blocking, which is typical in client–server systems including those implemented under the Enterprise JavaBean (EJB) specification. The novel feature is that tasks must wait for one of a number of parallel queues to clear its outstanding work. Thus, blocking time is the minimum of sojourn times at the parallel queues. Under certain simplifying assumptions, we solve this model for the probability distribution of blocking time and obtain a simple formula for its mean value. We then use this result in an aggregate server model of a larger queueing network in which further non-standard techniques are included to represent this form of blocking. We compare our approximate results against simulation data, obtaining good agreement for both system throughput and queue length probability distributions at equilibrium.     

Syntactic abstraction in scheme

Naive program transformations can have surprising effects due to the interaction between introduced identifier references and previously existing identifier bindings, or between introduced bindings and previously existing references. These interactions can result in inadvertent binding, or capturing, of identifiers. A further complication is that transformed programs may have little resemblance to original programs, making correlation of source and object code difficult. This article describes an efficient macro system that prevents inadvertent capturing while maintaining the correlation between source and object code. The macro system allows the programmer to define program transformations using an unrestricted, general-purpose language. Previous approaches to the capturing problem have been inadequate, overly restrictive, or inefficient, and the problem of source-object correlation has been largely unaddressed. The macro system is based on a new algorithm for implementing syntactic transformations and a new representation for syntactic expressions.This material is based on work supported by the National Science Foundation under grant number CCR-8803432.Robert Hieb died in an automobile accident in April 1992.     

Bridging abstraction layers in process mining

While the maturity of process mining algorithms increases and more process mining tools enter the market, process mining projects still face the problem of different levels of abstraction when comparing events with modeled business activities. Current approaches for event log abstraction try to abstract from the events in an automated way that does not capture the required domain knowledge to fit business activities. This can lead to misinterpretation of discovered process models. We developed an approach that aims to abstract an event log to the same abstraction level that is needed by the business. We use domain knowledge extracted from existing process documentation to semi-automatically match events and activities. Our abstraction approach is able to deal with n:m relations between events and activities and also supports concurrency. We evaluated our approach in two case studies with a German IT outsourcing company.     

Markovian bulk service queue with delayed vacations

An M/M(a, b)/1 queueing system with multiple vacations is studied, in which if the number of customers in the queue is a - 1 either at a service completion epoch or at a vacation completion point, the server will wait for an exponential time in the system which is called the changeover time. During this changeover time if there is an arrival the server will start service immediately, otherwise at the end of the changeover time the server will go for a vacation. The duration of vacation is also exponential. This paper is concerned with the determination of the stationary distribution of the number of customers in the queue and the waiting time distribution of an arriving customer. The expected queue length is also obtained. Sample numerical illustrations are given.     

FFD bin packing for item sizes with uniform distributions on [0, 1/2]

We study the expected behavior of the FFD bin-packing algorithm applied to items whose sizes are distributed in accordance with a Poisson process with rateN on the interval[0, 1/2] of item sizes. By viewing the algorithm as a succession of queueing processes we show that the expected wasted space for FFD bin-packing is bounded above by 11.3 bins, independent ofN.     

Contextualization as an independent abstraction mechanism for conceptual modeling

The notion of context appears in computer science, as well as in several other disciplines, in various forms. In this paper, we present a general framework for representing the notion of context in information modeling. First, we define a context as a set of objects, within which each object has a set of names and possibly a reference: the reference of the object is another context which “hides” detailed information about the object. Then, we introduce the possibility of structuring the contents of a context through the traditional abstraction mechanisms, i.e., classification, generalization, and attribution. We show that, depending on the application, our notion of context can be used as an independent abstraction mechanism, either in an alternative or a complementary capacity with respect to the traditional abstraction mechanisms. We also study the interactions between contextualization and the traditional abstraction mechanisms, as well as the constraints that govern such interactions. Finally, we present a theory for contextualized information bases. The theory includes a set of validity constraints, a model theory, as well as a set of sound and complete inference rules. We show that our core theory can be easily extended to support embedding of particular information models in our contextualization framework.     

Finite horizon analysis of Markov Chains with the Murϕ verifier

In this paper we present an explicit disk-based verification algorithm for Probabilistic Systems defining discrete time/finite state Markov Chains. Given a Markov Chain and an integer k (horizon), our algorithm checks whether the probability of reaching an error state in at most k steps is below a given threshold. We present an implementation of our algorithm within a suitable extension of the Murϕ verifier. We call the resulting probabilistic model checker FHP-Murϕ (Finite Horizon ProbabilisticMurϕ). We present experimental results comparing FHP-Murϕ with (a finite horizon subset of) PRISM, a state-of-the-art symbolic model checker for Markov Chains. Our experimental results show that FHP-Murϕ can handle systems that are out of reach for PRISM, namely those involving arithmetic operations on the state variables (e.g. hybrid systems). This research has been partially supported by MURST projects MEFISTO and SAHARA. This paper is a journal version of the conference paper [16].     

Stochastic system controller synthesis for reachability specifications encoded by random sets

We consider a reach–avoid specification for a stochastic hybrid dynamical system defined as reaching a goal set at some finite time, while avoiding an unsafe set at all previous times. In contrast with earlier works which consider the target and avoid sets as deterministic, we consider these sets to be probabilistic. An optimal control policy is derived which maximizes the reach–avoid probability. Special structure on the stochastic sets is exploited to make the computation tractable for large space dimensions.     

State explosion in almost-sure probabilistic reachability

We show that the problem of reaching a state set with probability 1 in probabilistic-nondeterministic systems operating in parallel is EXPTIME-complete. We then show that this probabilistic reachability problem is EXPTIME-complete also for probabilistic timed automata.     

Conditioned asymptotics for tail probabilities in large multiplexers

Consider a buffer whose input is a superposition of L independent identical sources, and which is served at rate sL. Recent work has shown that, under very general circumstances, the stationary tail probabilities for the queue of unfinished work Q in the buffer have the asymptotics P[Q > Lb] ≈ e^−LI(b) for large L. Here the shape function, I, is obtained from a variational expression involving the transient log cumulant generating function of the arrival process.

In this paper, we extend this analysis to cover time-dependent asymptotics for Markov arrival processes subject to conditioning at some instant. In applications we envisage that such conditioning would arise due to knowledge of the queue at a coarse-grained level, for example of the number of current active sources. We show how such partial knowledge can be used to predict future tail probabilities by use of a time dependent, conditioned shape function. We develop some heuristics to describe the time-dependent shape function in terms of a reduced set of quantities associated with the underlying arrivals process and show how to calculate them for renewal arrivals and a class of ON-OFF arrivals. This bypasses the full variational calculation of the shape function for such models.     

Automatic abstraction in reinforcement learning using data mining techniques

In this paper, we used data mining techniques for the automatic discovering of useful temporal abstraction in reinforcement learning. This idea was motivated by the ability of data mining algorithms in automatic discovering of structures and patterns, when applied to large data sets. The state transitions and action trajectories of the learning agent are stored as the data sets for data mining techniques. The proposed state clustering algorithms partition the state space to different regions. Policies for reaching different parts of the space are separately learned and added to the model in a form of options (macro-actions). The main idea of the proposed action sequence mining is to search for patterns that occur frequently within an agent’s accumulated experience. The mined action sequences are also added to the model in a form of options. Our experiments with different data sets indicate a significant speedup of the Q-learning algorithm using the options discovered by the state clustering and action sequence mining algorithms.     

Proving Approximate Implementations for Probabilistic I/O Automata

In this paper we introduce the notion of approximate implementations for Probabilistic I/O Automata (PIOA) and develop methods for proving such relationships. We employ a task structure on the locally controlled actions and a task scheduler to resolve nondeterminism. The interaction between a scheduler and an automaton gives rise to a trace distribution—a probability distribution over the set of traces. We define a PIOA to be a (discounted) approximate implementation of another PIOA if the set of trace distributions produced by the first is close to that of the latter, where closeness is measured by the (resp. discounted) uniform metric over trace distributions. We propose simulation functions for proving approximate implementations corresponding to each of the above types of approximate implementation relations. Since our notion of similarity of traces is based on a metric on trace distributions, we do not require the state spaces nor the space of external actions of the automata to be metric spaces. We discuss applications of approximate implementations to verification of probabilistic safety and termination.