Streaming data analytics via message passing with application to graph algorithms

The need to process streaming data, which arrives continuously at high-volume in real-time, arises in a variety of contexts including data produced by experiments, collections of environmental or network sensors, and running simulations. Streaming data can also be formulated as queries or transactions which operate on a large dynamic data store, e.g. a distributed database.     

A fast multiprocessor message passing implementation

We present the design and implementation of a loosely coupled multiprocessor built from off-the-shelf parts. Message passing is used as the communication paradigm. Several novel techniques are used to reduce the demands on the kernel from the message passing subsystem. We achieve message passing times of the same order for messages within processors and interprocessor messages, allowing transparent interprocess communication. Because it is possible to achieve these performance results, we conclude that process allocation need not be a critical problem in efficient multiprocessor design, at least for small scale multiprocessors.     

分布估计算法研究进展

作为一种新颖的基于概率模型的进化算法,近年来分布估计算法（EDA）得到了广泛的研究和发展.在介绍分布估计算法原理和特点的基础上,重点综述了近些年分布估计算法的研究进展,包括改进概率模型、保持种群多样性以及设计混合算法,进而总结了分布估计算法在理论及应用方面的研究现状,最后提出了有待进一步研究的若干方向和内容.     

A message passing approach to robot programming

We are developing a communication oriented interactive environment for robot programming. A set of independent software modules for robot motoring control, low level sensoring processing, kinematics simulation and world model construction has been implemented as separate tools. They have been merged in a software environment whose implementation is based on the operating system Accent and can be accessed through an advanced graphics workstation. Each module interacts with the others through message exchange; each module may be organized into new submodules as new operational behaviours belonging to its functional area become necessary.     

A message passing benchmark for unbalanced applications

We present a distributed memory parallel implementation of the unbalanced tree search (UTS) benchmark using MPI and investigate MPI’s ability to efficiently support irregular and nested parallelism through continuous dynamic load balancing. Two load balancing methods are explored: work sharing using a centralized work server and distributed work stealing using explicit polling to service steal requests. Experiments indicate that in addition to a parameter defining the granularity of load balancing, message-passing paradigms require additional techniques to manage the volume of communication and mitigate runtime overhead. Using additional parameters, we observed an improvement of up to 3–4X in parallel performance. We report results for three distributed memory parallel computer systems and use UTS to characterize the performance and scalability on these systems. Overall, we find that the simpler work sharing approach with a single work server achieves good performance on hundreds of processors and that our distributed work stealing implementation scales to thousands of processors and delivers more robust performance that is less sensitive to the particular workload and load balancing parameters.     

A modal logic for message passing processes

A first-order modal logic is given for describing properties of processes which may send and receive values or messages along communication ports. We give two methods for proving that a process enjoys such a property. The first is the construction, for each processP and formulaF, of acharacteristic formula P satF such thatP enjoys the propertyF if and only if the formulaP satF is logically equivalent to tt. The second is a sound and complete proof system whose judgements take the formB P: F, meaning: under the assumptionB the processP enjoys the propertyF.The notion ofsymbolic operational semantics plays a crucial role in the design of both the characteristic formulae and the proof system.This work was been supported by the SERC grant GR/H16537 and the ESPRIT BRA CONCUR II     

Regularized continuous estimation of distribution algorithms

Regularization is a well-known technique in statistics for model estimation which is used to improve the generalization ability of the estimated model. Some of the regularization methods can also be used for variable selection that is especially useful in high-dimensional problems. This paper studies the use of regularized model learning in estimation of distribution algorithms (EDAs) for continuous optimization based on Gaussian distributions. We introduce two approaches to the regularized model estimation and analyze their effect on the accuracy and computational complexity of model learning in EDAs. We then apply the proposed algorithms to a number of continuous optimization functions and compare their results with other Gaussian distribution-based EDAs. The results show that the optimization performance of the proposed RegEDAs is less affected by the increase in the problem size than other EDAs, and they are able to obtain significantly better optimization values for many of the functions in high-dimensional settings.     

Drift and scaling in estimation of distribution algorithms

This paper considers a phenomenon in Estimation of Distribution Algorithms (EDA) analogous to drift in population genetic dynamics. Finite population sampling in selection results in fluctuations which get reinforced when the probability model is updated. As a consequence, any probability model which can generate only a single set of values with probability 1 can be an attractive fixed point of the algorithm. To avoid this, parameters of the algorithm must scale with the system size in strongly problem-dependent ways, or the algorithm must be modified. This phenomenon is shown to hold for general EDAs as a consequence of the lack of ergodicity and irreducibility of the Markov chain on the state of probability models. It is illustrated in the case of UMDA, in which it is shown that the global optimum is only found if the population size is sufficiently large. For the needle-in-a haystack problem, the population size must scale as the square-root of the size of the search space. For the one-max problem, the population size must scale as the square-root of the problem size.     

Space complexity of estimation of distribution algorithms

In this paper, we investigate the space complexity of the Estimation of Distribution Algorithms (EDAs), a class of sampling-based variants of the genetic algorithm. By analyzing the nature of EDAs, we identify criteria that characterize the space complexity of two typical implementation schemes of EDAs, the factorized distribution algorithm and Bayesian network-based algorithms. Using random additive functions as the prototype, we prove that the space complexity of the factorized distribution algorithm and Bayesian network-based algorithms is exponential in the problem size even if the optimization problem has a very sparse interaction structure.     

Parallel adaptive wavefront algorithms solving Lyapunov equations for the Cholesky factor on message passing multiprocessors

The order of the matrices involved in several algebraic problems decreases during the solution process. In these cases, parallel algorithms which use adaptive solving block sizes offer better performance results than the ones obtained on parallel algorithms using traditional constant block sizes. Recently, new parallel wavefront algorithms solving the Lyapunov equations for the Cholesky factor using Hammarling's method on message passing multiprocessors systems have been designed. In this paper, new parallel adaptive versions of these parallel algorithms are described, and experimental results obtained on an SGI Power Challenge and a SUN UltraSparc cluster are presented. Copyright © 1999 John Wiley & Sons, Ltd.     

Adaptive checkpointing in message passing distributed systems

Determining consistent global checkpoints is common to many distributed problems such as fault-tolerance, distributed debugging, properties detection, etc. Uncoordinated and coordinated checkpointing algorithms have been traditionally used for such determinations. This paper addresses a third technique, namely adaptive checkpointing, that has recently emerged. This technique assumes processes take local checkpoints independently and requires them to take additional local checkpoints in order that all local checkpoints be members of some consistent global checkpoint. We first study the characteristics of such adaptive algorithms. Then, a general adaptive checkpointing algorithm is designed from a condition, first stated by Netzer and Xu, that answers the following question: ‘does a given local checkpoint belong to a consistent global checkpoint’' (such a local checkpoint is not useless). The resulting algorithm has the nice property to reduce the number of additional local checkpoints taken to ensure the property ‘no local checkpoint is useless’. Futhermore, it provides each local checkpoint with a consistent global checkpoint to which it belongs. Compared to uncoordinated and coordinated checkpointing algorithms, this algorithm combines the advantages of both without inheriting their drawbacks.     

Hiding message delivery latency using Direct-to-Cache-Transfer techniques in message passing environments

Communication overhead is the key obstacle to reaching hardware performance limits. The majority is associated with software overhead, a significant portion of which is attributed to message copying. To reduce this copying overhead, we have devised techniques that do not require to copy a received message in order for it to be bound to its final destination. Rather, a late-binding mechanism, which involves address translation and a dedicated cache, facilitates fast access to received messages by the consuming process/thread.We have introduced two policies namely Direct to Cache Transfer (DTCT) and lazy DTCT that determine whether a message after it is bound needs to be transferred into the data cache. We have studied the proposed methods in simulation and have shown their effectiveness in reducing access times to message payloads by the consuming process.     

Generalized communicators in the message passing interface

We propose extensions to the message passing interface (MPI) that generalize the MPI communicator concept to allow multiple communication endpoints per process, dynamic creation of endpoints, and the transfer of endpoints between processes. The generalized communicator construct can be used to express a wide range of interesting communication structures, including collective communication operations involving multiple threads per process, communications between dynamically created threads or processes, and object-oriented applications in which communications are directed to specific objects. Furthermore, this enriched functionality can be provided in a manner that preserves backward compatibility with MPI. We describe the proposed extensions, illustrate their use with examples, and describe a prototype implementation in the popular MPI implementation MPICH     

A new initialization procedure for the distributed estimation of distribution algorithms

Estimation of distribution algorithms (EDAs) are one of the most promising paradigms in today’s evolutionary computation. In this field, there has been an incipient activity in the so-called parallel estimation of distribution algorithms (pEDAs). One of these approaches is the distributed estimation of distribution algorithms (dEDAs). This paper introduces a new initialization mechanism for each of the populations of the islands based on the Voronoi cells. To analyze the results, a series of different experiments using the benchmark suite for the special session on Real-parameter Optimization of the IEEE CEC 2005 conference has been carried out. The results obtained suggest that the Voronoi initialization method considerably improves the performance obtained from a traditional uniform initialization.     

Toward formally-based design of message passing programs

Presents a systematic approach to the development of message passing programs. Our programming model is SPMD, with communications restricted to collective operations: scan, reduction, gather, etc. The design process in such an architecture-independent language is based on correctness-preserving transformation rules that are provable in a formal functional framework. We develop a set of design rules for composition and decomposition. For example, scan followed by reduction is replaced by a single reduction, and global reduction is decomposed into two faster operations. The impact of the design rules on the target performance is estimated analytically and tested in machine experiments. As a case study, we design two provably correct, efficient programs using the Message Passing Interface (MPI) for the famous maximum segment sum problem, starting from an intuitive, but inefficient, algorithm specification     

On the interconnection of message passing systems

One of the most important abstractions for designing distributed programs is the broadcast facility. In this paper, we study the interconnection of distributed message passing systems. We have shown that totally ordered systems cannot be properly interconnected in any form. However, we have provided a simple protocol to properly interconnect FIFO ordered systems.     

Priority inheritance and message passing communication: A formal treatment

Priority is one means of representing scheduling information in a concurrent real-time programming language. Unfortunately a static priority scheme can give rise to inversion when combined with synchronization primitives. In this paper one means of overcoming inversion, namely priority inheritance, is specified together with the semantics of a simple message-based language. The formal notation Z is used for this treatment. Following the specification, investigations are made of the operational behavior of programs that adhere to these semantics.     

Predicate transfer and timeout in message passing systems

Timeout in a message passing system is studied with emphasis on the notion of predicate transfer. Under certain conditions, a predicate describing the state of the sender can be transferred to the receiver when a timeout occurs. These conditions relate to the way the message passing system is implemented. This relationship is discussed and a technique for formally describing the semantics of message passing with timeout is presented.     

A method of program transformation between variable sharing and message passing

There have been proposed and used many programming concepts for realizing interprocess communication and synchronization, and two fundamental categories among them are variable sharing (e.g. semaphores) and message passing (signals, input/output in CSP and redezvous in Ada). The aim of this paper is to investigate the relationship between these two mechanisms, and in particular, program transformation between them. We here propose a transformation method between parallel programs written in each of these two, which uses transformation schemes named monitor transformations. We exhibit the method by showing through a chain of program transformations the equivalence of two versions of a parallel algorithm called dynamic sorting array.