Coterie join algorithm

Given a set of nodes in a distributed system, a coterie is a collection of subsets of the set of nodes such that any two subsets have a nonempty intersection and are not properly contained in one another. A subset of nodes in a coterie is called a quorum. An algorithm, called the join algorithm, which takes nonempty coteries as input, and returns a new, larger coterie called a composite coterie is introduced. It is proved that a composite coterie is nondominated if and only if the input coteries are nondominated. Using the algorithm, dominated or nondominated coteries may be easily constructed for a large number of nodes. An efficient method for determining whether a given set of nodes contains a quorum of a composite coterie is presented. As an example, tree coteries are generalized using the join algorithm, and it is proved that tree coteries are nondominated. It is shown that the join algorithm may be used to generate read and write quorums which may be used by a replica control protocol     

Minimizing the maximum delay for reaching consensus in quorum-basedmutual exclusion schemes

The use of quorums is a well-known approach to achieving mutual exclusion in distributed computing systems. This approach works based on a coterie, a special set of node groups where any pair of the node groups shares at least one common node. Each node group in a coterie is called a quorum. Mutual exclusion is ensured by imposing that a node gets consensus from all nodes in at least one of the quorums before it enters a critical section. In a quorum-based mutual exclusion scheme, the delay for reaching consensus depends critically on the coterie adopted and, thus, it is important to find a coterie with small delay. Fu (1997) introduced two related measures called max-delay and mean-delay. The former measure represents the largest delay among all nodes, while the latter is the arithmetic mean of the delays. She proposed polynomial-time algorithms for finding max-delay and mean-delay optimal coteries when the network topology is a tree or a ring. In this paper, we first propose a polynomial-time algorithm for finding max-delay optimal coteries and, then, modify the algorithm so as to reduce the mean-delay of generated coteries. Unlike the previous algorithms, the proposed algorithms can be applied to systems with arbitrary topology     

Nondominated coteries on graphs

Let C and D be two distinct coteries under the vertex set V of a graph G=(V,E) that models a distributed system. Coterie C is said to G-dominate D (with respect to G) if the following condition holds: For any connected subgraph H of G that contains a quorum in D (as a subset of its vertex set), there exists a connected subgraph H' of H that contains a quorum in C. A coterie C on a graph G is said to be G-nondominated (G-ND) (with respect to G) if no coterie D(≠C) on G G-dominates C. Intuitively, a G-ND coterie consists of irreducible quorums. This paper characterizes G-ND coteries in graph theoretical terms, and presents a procedure for deciding whether or not a given coterie C is G-ND with respect to a given graph G, based on this characterization. We then improve the time complexity of the decision procedure, provided that the given coterie C is nondominated in the sense of Garcia-Molina and Barbara (1985). Finally, we characterize the class of graphs G on which the majority coterie is G-ND     

Composite k-arbiters

The k-arbiter is a useful concept to solve the distributed h-out-of-k mutual exclusion problem. The distributed h-out-of-k mutual exclusion algorithms, based on the k-arbiter, have the benefits of high fault tolerance and low message cost. However, according to the definition of the k-arbiter, it is required to have a nonempty intersection among any (κ + 1) quorums in a k-arbiter. Consequently, constructing k-arbiters is difficult. The coterie join operation proposed by Neilsen and Mizuno (1992) produces a new and larger coterie by joining known coteries. By extending the coterie join operation, we first propose a k-arbiter join operation to construct a new and larger k-arbiter from known k-arbiters for a large system. Then, we derive a necessary and sufficient condition for the k-arbiter join operation to construct a nondominated joined k-arbiter. Moreover, we discuss availability properties of the joined k-arbiters. We observe that, by selecting proper k-arbiters, the joined k-arbiter can provide a higher availability than that of the original input. Finally, we propose a k-arbiter compound, operation to construct k-arbiters by using coteries and/or k-coteries. By that way, the problem of constructing k-arbiters can be reduced to the problem of constructing coteries and/or k-coteries     

A Token-Based Distributed Group Mutual Exclusion Algorithm with Quorums

The group mutual exclusion problem is a generalization of mutual exclusion problem such that a set of processes in the same group can enter critical section simultaneously. In this paper, we propose a distributed algorithm for the group mutual exclusion problem in asynchronous message passing distributed systems. Our algorithm is based on tokens, and a process that obtains a token can enter critical section. For reducing message complexity, it uses coterie as a communication structure when a process sends a request messages. Informally, coterie is a set of quorums, each of which is a subset of the process set, and any two quorums share at least one process. The message complexity of our algorithm is $O(|Q|)$ in the worst case, where $|Q|$ is a quorum size that the algorithm adopts. Performance of the proposed algorithm is presented by analysis and discrete event simulation. Especially, the proposed algorithm achieves high concurrency, which is a performance measure for the number of processes that can be in critical section simultaneously.     

Recognizing nondominated coteries and wr-coteries by availability

Coterie is a widely accepted concept for solving the mutual exclusion problem. Nondominated coteries are an important class of coteries which have better performance than dominated coteries. The performance of a coterie is usually measured by availability. Higher availability of a coterie exhibits greater ability to tolerate node or communication link failures. In this paper, we demonstrate a way to recognize nondominated coteries using availability. By evaluating the availability of a coterie instead of using a formal proof, the coterie can be recognized as a nondominated coterie or not. Moreover, with regard to wr-coterie, a concept for solving the replica control problem, we also present a similar result for recognizing nondominated wr-coteries. Finally, we apply our results to some well-known coteries and wr-coteries     

Generating and approximating nondominated coteries

A coterie, which is used to realize mutual exclusion in a distributed system is a family C of incomparable subsets such that every pair of subsets in C has at least one element in common. Associate with a family of subsets C a positive (i.e., monotone) Boolean function f_c such that f_c(x)=1 if the Boolean vector x is equal to or greater than the characteristic vector of some subset in C, and 0 otherwise. It is known that C is a coterie if and only if f_c is dual-minor, and is a nondominated (ND) coterie if and only if f_c is self-dual. We introduce an operator ρ, which transforms a positive self-dual function into another positive self-dual function, and the concept of almost-self-duality, which is a close approximation to self-duality and can be checked in polynomial time (the complexity of checking positive self-duality is currently unknown). After proving several interesting properties of them, we propose a simple algorithm to check whether a given positive function is self-dual or not. Although this is not a polynomial algorithm, it is practically efficient in most cases. Finally, we present an incrementally polynomial algorithm that generates all positive self-dual functions (ND coteries) by repeatedly applying p operations. Based on this algorithm, all ND coteries of up to seven variables are computed     

A geometric approach for constructing coteries and k-coteries

Quorum-based mutual exclusion algorithms are resilient to node and communication line failures. Recently, some mutual exclusion algorithms successfully use logical structures to construct coteries with small quorums sizes. In this paper, we introduce a geometric approach on dealing with the logical structures and present some useful geometric properties for constructing coteries and k-coteries. Based on those geometric properties, a logical structure named three-sided graph is proposed to provide a new scheme for constructing coteries with small quorums: The smallest quorum size is O(√N) in a homogeneous system with N nodes and O(1) in a heterogeneous system. In addition, we also extend the three-sided graph to the O-sided graph for constructing k-coteries     

Refined quorum systems

It is considered good distributed computing practice to devise object implementations that tolerate contention, periods of asynchrony and a large number of failures, but perform fast if few failures occur, the system is synchronous and there is no contention. This paper initiates the first study of quorum systems that help design such implementations by encompassing, at the same time, optimal resilience, as well as optimal best-case complexity. We introduce the notion of a refined quorum system (RQS) of some set S as a set of three classes of subsets (quorums) of S: first class quorums are also second class quorums, themselves being also third class quorums. First class quorums have large intersections with all other quorums, second class quorums typically have smaller intersections with those of the third class, the latter simply correspond to traditional quorums. Intuitively, under uncontended and synchronous conditions, a distributed object implementation would expedite an operation if a quorum of the first class is accessed, then degrade gracefully depending on whether a quorum of the second or the third class is accessed. Our notion of refined quorum system is devised assuming a general adversary structure, and this basically allows algorithms relying on refined quorum systems to relax the assumption of independent process failures, often questioned in practice. We illustrate the power of refined quorums by introducing two new optimal Byzantine-resilient distributed object implementations: an atomic storage and a consensus algorithm. Both match previously established resilience and best-case complexity lower bounds, closing open gaps, as well as new complexity bounds we establish here. Each of our algorithms is representative of a different class of architectures, highlighting the generality of the refined quorum abstraction.     

Dynamic grid quorum: a reconfigurable grid quorum and its power optimization algorithm

Power-saving has become a central issue for well-configured SOC platforms. In particular, as a high percentage of the total energy is used by the storage systems, the cost effectiveness of data management is equally as important as reliability and availability. To address this issue, we propose the dynamic grid quorum as a method for reducing the power consumption of large-scale distributed storage systems. The basic principle of our approach is to skew the workload toward a small number of quorums. This can be realized using the following three techniques. First, our system allows reconfiguration by exchanging nodes without any data migration, so that high-capacity nodes can be reallocated to busier quorums. Second, for more effective skewing of the workload, we introduce the notion of dual allocation, which makes it possible to consider two distinct allocations in the same grid for write and read quorums. Finally, we present an optimization algorithm to find a pair of a strategy and an allocation of nodes, which minimizes power for a given system setting and its workload. We also demonstrate that the dynamic grid quorum saves, on average, 14–25% energy compared with static configurations, when the intensity of the total workload changes.     

Access cost for asynchronous Byzantine quorum systems

Summary. Quorum systems have been used to implement many coordination problems in distributed systems. In this paper, we study the cost of accessing quorums in asynchronous systems. We formally define the asynchronous access cost of quorum systems and argue that the asynchronous access cost and not the size of a quorum is the right measure of message complexity of protocols using quorums in asynchronous systems. We show that previous quorum systems proposed in the literature have a very high asynchronous access cost. We propose a reformulation of the definition of Byzantine quorum systems that captures the requirement for non-blocking access to quorums in asynchronous systems. We present new Byzantine quorum systems with low asynchronous access cost whose other performance parameters match those of the best Byzantine quorum systems proposed in the literature. In particular, we present a construction for the disjoint failure pattern that outperforms previously proposed systems for that pattern. Received: September 1999 / Accepted: September 2000     

Synchronous Byzantine quorum systems

Summary. Quorum systems have been used to implement many coordination problems in distributed systems such as mutual exclusion, data replication, distributed consensus, and commit protocols. Malkhi and Reiter recently proposed quorum systems that can tolerate Byzantine failures; they called these systems Byzantine quorum systems and gave some examples of such quorum systems. In this paper, we propose a new definition of Byzantine quorums that is appropriate for synchronous systems. We show how these quorums can be used for data replication and propose a general construction of synchronous Byzantine quorums using standard quorum systems. We prove tight lower bounds on the load of synchronous Byzantine quorums for various patterns of failures and we present synchronous Byzantine quorums that have optimal loads that match the lower bounds for two failure patterns. Received: June 1998 / Accepted: August 1999     

Probabilistic quorums for dynamic systems

A quorum system is a set of sets such that every two sets in the quorum system intersect. Quorum systems are well known building blocks for maintaining information in a distributed system while providing high availability and good load balance. Probabilistic Quorum Systems (PQS) are variants of quorum systems that relax the strict intersection requirement. They are particularly attractive for large scale systems due to their simplicity and highly efficient availability—load balance tradeoff. We introduce scalable techniques that maintain a PQS in a highly decentralized and highly dynamic setting. We address two challenges. First we show how PQS can be realized efficiently even when each process maintains knowledge of only a constant number of other members. Second, we provide algorithms that adaptively evolve the quorums to adjust to the changes in the system caused by processes leaving and joining the system over time.     

Crumbling walls: a class of practical and efficient quorum systems

Summary.  A quorum system is a collection of sets (quorums) every two of which intersect. Quorum systems have been used for many applications in the area of distributed systems, including mutual exclusion, data replication and dissemination of information. In this paper we introduce a general class of quorum systems called Crumbling Walls and study its properties. The elements (processors) of a wall are logically arranged in rows of varying widths. A quorum in a wall is the union of one full row and a representative from every row below the full row. This class considerably generalizes a number of known quorum system constructions. The best crumbling wall is the CWlog quorum system. It has small quorums, of size O(lg n), and structural simplicity. The CWlog has optimal availability and optimal load among systems with such small quorum size. It manifests its high quality for all universe sizes, so it is a good choice not only for systems with thousands or millions of processors but also for systems with as few as 3 or 5 processors. Moreover, our analysis shows that the availability will increase and the load will decrease at the optimal rates as the system increases in size. Received: August 1995 / Accepted: August 1996     

On multilevel voting

Summary In a distributed system mutual exclusion is often used to maintain consistency when restricted operations are performed. Mechanisms guaranteeing mutual exclusions should be both resilient and efficient. Resiliency implies high resource availability in the face of failures, while efficiency implies low overhead incurred by performing restricted operations. In this paper, we propose and study a general paradigm, called multilevel voting, which provides a general framework to assist in the design of resilient and efficient mutual exclusion mechanisms. The proposed method uses multiple level quorum consensus. Unlike another method based on the use of multiple quorum consensus, the proposed model only contains one type of integrity constraints. This has the benefit of being conceptually simple and easy to reason about. The strong resemblance with the traditional quorum consensus makes it easy for the proposed paradigm to embed any technique based on traditional quorum schemes. We show that the proposed model represents the exact class of coteries. This means that not only does it have all the power of coteries, but also all schemes under the model are correct. Thus, should the need arise, we can interchange two schemes freely without using any extra mechanisms to ensure correctness. We study a number of issues that have impact on performance such as the degree of a multilevel scheme and the order of a coterie. We explain how the model can be extended also to model the schemes for the synchronization of read and write of replicated data. We provide algorithms for the design of multilevel schemes in the context of mutual exclusion and that of read and write of replicated data. J. Tang received the M.S. degree in computer science from the University of Iowa in 1983 and the Ph.D degree in computer science from the Pennsylvania State University in 1988. Since 1988, he has been an assistant professor with the Department of Computer Science, Memorial University of Newfoundland, St. John's, Newfoundland, Canada. His current research interests include distributed computing, fault tolerance in distributed systems, database modeling and multidatabase systems.This work was supported in part by the Natural Sciences and Engineering Research Council of Canada, individual operating grant OGP0041916     

(h,k)-Arbiters for h-out-of-k mutual exclusion problem

h-Out-of-k mutual exclusion is a generalization of the 1-mutual exclusion problem, where there are k units of shared resources and each process requests h (1hk) units at the same time. Though k-arbiter has been shown to be a quorum-based solution to this problem, quorums in k-arbiter are much larger than those in the 1-coterie for 1-mutual exclusion. Thus, the algorithm based on k-arbiter needs many messages. This paper introduces the new notion that each request uses different quorums depending on the number of units of its request. Based on the notion, this paper defines two (h,k)-arbiters for h-out-of-k mutual exclusion: a uniform (h,k)-arbiter and a (k+1)-cube (h,k)-arbiter. The quorums in each (h,k)-arbiter are not larger than the ones in the corresponding k-arbiter; consequently, it is more efficient to use (h,k)-arbiters than the k-arbiters. A uniform (h,k)-arbiter is a generalization of the majority coterie for 1-mutual exclusion. A (k+1)-cube (h,k)-arbiter is a generalization of square grid coterie for 1-mutual exclusion.     

Delay optimal coteries on regular symmetric networks

Coteries, introduced by Garcia-Molina and Barbara [Journal for the Association for Computing Machinery, 32 (4) (1985) 841], are an important and effective tool for enforcing mutual exclusion in distributed systems. Communication delay is an important performance measure for a coterie. Fu et al. [IEEE Transactions on Parallel and Distributed Systems, 8 (1) (1997) 59] emphasize that while calculating communication delay, the actual distances between different sites in a network must be taken into account and using this idea, obtain delay optimal coteries for trees, rings and hypercubes. Also, topology of an interconnection network plays an important role in the performance of a distributed system. For certain applications, it is desirable that the degree of each node in the interconnection network is the same. Constant Degree Four Cayley Graphs introduced by Vadapalli and Srimani [IEEE Transactions on Parallel and Distributed Systems 7(2) (1996) 26] provide an ideal topology for such applications. They are regular, have a logarithmic diameter and a node connectivity of four. In this paper, we prove that no coterie on an arbitrary network can have a delay of less than half the diameter of the network and use this result to obtain delay optimal coteries on regular symmetric networks with special reference to constant degree four Cayley interconnection-network.     

分布式系统数据复制协议的研究

数据复制正在并行信息系统的设计中起着越来越重要的作用.特别地,为了提高性能和可用性,对集群体系结构的广泛使用经常要求复制数据.然而,维持不同拷贝的一致性带来严重的可伸缩性问题.为了克服这个局限性,基于表决的协议经常被作为一种缩减复制的总开销的方法.为了更好理解它们在实际中的性能,分析了几种表决算法,结果是基于表决的复制协议表现出不错的性能.测试表明ROWA-A(read-one/write-all-available)协议对于需要大量数据复制的应用是最好的选择.     

Average probe complexity in quorum systems

This paper discusses the probe complexity of randomized algorithms and the deterministic average case probe complexity for some classes of nondominated coteries, including majority, crumbling walls, tree, wheel and hierarchical quorum systems, and presents upper and lower bounds for the probe complexity of quorum systems in these classes.     

Analysis of quorum-based protocols for distributed (k+1)-exclusion

A generalization of the majority quorum for the solution of the distributed (k+1)-exclusion problem is proposed. This scheme produces a family of quorums of varying sizes and availabilities indexed by integral divisors r of k. The cases r=1 and r=k correspond to known majority based quorum generation algorithms MAJ and DIV, whereas intermediate values of r interpolate between these two extremes. A cost and availability analysis of the proposed methods is also presented. An interesting implication of this analysis is that in a reasonably reliable environment with a large number of sites, even protocols with low communication costs attain high availability