Using refinement calculus techniques to prove linearizability

Stepwise refinement is a method for systematically transforming a high-level program into an efficiently executable one. A sequence of successively refined programs can also serve as a correctness proof, which makes different mechanisms in the program explicit. We present rules for refinement of multi-threaded shared-variable concurrent programs. We apply our rules to the problem of verifying linearizability of concurrent objects, that are accessed by an unbounded number of concurrent threads. Linearizability is an established correctness criterion for concurrent objects, which states that the effect of each method execution can be considered to occur atomically at some point in time between its invocation and response. We show how linearizability can be expressed in terms of our refinement relation, and present rules for establishing this refinement relation between programs by a sequence of local transformations of method bodies. Contributions include strengthenings of previous techniques for atomicity refinement, as well as an absorption rule, which is particularly suitable for reasoning about concurrent algorithms that implement atomic operations. We illustrate the application of the refinement rules by proving linearizability of Treiber’s concurrent stack algorithm and Michael and Scott’s concurrent queue algorithm.     

Sequentially consistent versus linearizable counting networks

We compare the impact of timing conditions on implementing sequentially consistent and linearizable counters using (uniform) counting networks in distributed systems. For counting problems in application domains which do not require linearizability but will run correctly if only sequential consistency is provided, the results of our investigation, and their potential payoffs, are threefold: First, we show that sequential consistency and linearizability cannot be distinguished by the timing conditions previously considered in the context of counting networks; thus, in contexts where these constraints apply, it is possible to rely on the stronger semantics of linearizability, which simplifies proofs and enhances compositionality. Second, we identify local timing conditions that support sequential consistency but not linearizability; thus, we suggest weaker, easily implementable timing conditions that are likely to be sufficient in many applications. Third, we show that any kind of synchronization that is too weak to support even sequential consistency may violate it significantly for some counting networks; hence, we identify timing conditions that are to be totally ruled out for specific applications that rely critically on either sequential consistency or linearizability. A preliminary version of this work appears in the Proceedings of the 18th annual ACM symposium on principles of distributed computing (PODC 1999), pp. 133–142, May 1999. This work has been partially supported by the IST Program of the European Union under projects DELIS (contract number 001907) and AEOLUS (contract number 15964).     

DisCComp – A Formal Model for Distributed Concurrent Components

Most large-scaled software systems are structured in distributed components to manage complexity and have to cope with concurrent executed threads. System decomposition and concurrent flow of execution are orthogonal. A sound semantic model that is powerful enough to handle distributed concurrent components but also realistic enough to provide a foundation for component technologies actually in use is still missing. Therefore, the paper introduces such an operational semantics for distributed concurrent component-based systems. Based on this formal model, UML-based modeling techniques are introduced. Tool support for modeling, code generation, and system execution is provided.     

TSO内存模型下限界可线性化的可判定性研究

TSO-to-TSO可线性化、TSO-to-SC可线性化和TSO可线性化是Total Store Order（简称TSO）内存模型下可线性化的三个变种。在本文中我们提出了?-限界TSO-to-TSO可线性化和?-限界TSO可线性化,考察了?-限界TSO-to-TSO可线性化、?-限界TSO-to-SC可线性化和?-限界TSO可线性化的验证问题。它们分别是这三种可线性化的限界版本,都使用?-扩展历史,这样的扩展历史对应的执行有着限界数目（不超过?个）的函数调用、函数返回、调用刷出和返回刷出动作。?-扩展历史对应执行中的写动作数目是不限界的,进而执行中使用的存储缓冲区的大小也是不限界的,对应的操作语义是无穷状态迁移系统,所以三个限界版本可线性化的验证问题是不平凡的。 我们将定义在并发数据结构与顺序规约之间的?-限界TSO-to-TSO可线性化、?-限界TSO-to-SC可线性化和?-限界TSO可线性化的验证问题归约到?-扩展历史集合之间的TSO-to-TSO可线性化问题,从而以统一的方式验证了TSO内存模型下可线性化的三个限界版本。验证方法的关键步骤是判定一个并发数据结构是否有一个特定的?-扩展历史。我们证明了这个问题是可判定的,证明方法是将这一问题归约为已知可判定的易失通道机器的控制状态可达问题。本质上,这一归约将每一个函数调用或函数返回动作转化为写、刷出或cas（compare-and-swap）动作。在TSO-to-TSO可线性化的定义中,一个函数调用或函数返回动作会同时影响存储缓冲区和控制状态。为了模拟函数调用或函数返回动作对存储缓冲区的影响,我们在每个函数调用或函数返回动作之后立刻执行一个特定的写动作。这个写动作及其对应的刷出动作模拟了函数调用或函数返回动作对存储缓冲区的影响。我们引入观察者进程,为每个函数调用或函数返回动作“绑定”一个观察者进程的cas动作,以这种方式模拟了函数调用或函数返回动作对控制状态的影响。因此,我们证明了TSO内存模型下可线性化的这三个限界版本都是可判定的。我们进而证明了在TSO内存模型下判定可线性化的这三个限界版本的复杂度都在递归函数的Fast-Growing层级中。我们通过证明已知对应复杂度的单通道简单通道机器的可达问题和TSO内存模型下可线性化的三个限界版本可以互相归约得到这个结论。     

A Java/CORBA virtual machine architecture for remote execution of optimization solvers in heterogeneous networks

Virtual machines for remote execution are a useful tool for utilizing light user interfaces and intensive application cores in different physical machines connected through the Internet. In a virtual machine, application cores are distributed in a network. Specific locations, operating systems and hardware characteristics are hidden by virtual machines. They make it possible to use a PC to execute user interfaces and (a few) high‐performance computers for application cores. We present a Java/CORBA‐based brokerage platform that allows remote execution of optimization solvers from a client running on any platform. The system offers a dynamic library of available problem solvers, and a graphic interface to browse several defined properties and metadata on available solvers. In addition, an embedded file compression module to reduce data transfer time is included as a plug‐in feature of the proposed virtual machine. Analogous systems could be constructed for applications in which interaction traffic time is much lower than execution time. Copyright © 2001 John Wiley & Sons, Ltd.     

A semantic framework for automatic generation of computational workflows using distributed data and component catalogues

Computational workflows are a powerful paradigm to represent and manage complex applications, particularly in large-scale distributed scientific data analysis. Workflows represent application components that result in individual computations as well as their interdependences in terms of dataflow. Workflow systems use these representations to manage various aspects of workflow creation and execution for users, such as the automatic assignment of execution resources. This article describes an approach to automating a new aspect of the process: the selection of application components and data sources. We present a novel approach that enables users to specify varying degrees of detail and amount of constraints in a workflow request, including the specification of constraints on input, intermediate or output data in the workflow, abstract workflow component classes rather than specific component implementations, and generic reusable workflow templates that express a pre-defined combination of components. The algorithm elaborates the user request into a set of fully ground workflows with specific choices of data sources and codes to be used so that they can be submitted for mapping and execution. The algorithm searches through the space of possible candidate workflows by creating increasingly more specialized versions of the original template and eliminating candidates that violate constraints cumulated in the candidate workflow as components and data sources are selected. A novel feature of our approach is that it assumes a distributed architecture where data and component catalogues are separate from the workflow system. The algorithm explicitly poses queries to external catalogues, and therefore any reasoning regarding data or component properties is not assumed to occur within the workflow system. We describe our implementation of this approach in the Wings workflow system. This implementation uses the W3C Web Ontology Language and associated reasoners to implement the workflow system as well as the data and component catalogues. This research demonstrates the use of artificial intelligence techniques to support the kinds of automation envisioned by the scientific community for large-scale distributed scientific data analysis.     

数据并行计算：概念,模型与系统

一、引言并行计算,或者并行处理,指的是这样一种努力和相关的研究:利用多个具有计算能力的部件来共同完成一个计算工作,以获得比用一个部件来完成要快的效果。这显然是一个很自然的想法。历史地看,几乎是自从有了计算机,就有了并行处理的想法和实践。在80年代后期到90年代初期,以寻求对人类面临的若干重     

Coordination and communication issues in multi-agent expert system: concurrent configuration design advisor

The model of concurrent configuration design and the architecture of Concurrent Configuration Design Advisor (CCDA) are considered. The CCDA is developed as an open dynamic expert system of interacting agents. In the context of this article a configuration problem is defined as a decision-making procedure performed by concurrent processes (agents). The analysis of concurrent and sequential activities in this system is based on structural transformations (in the form of graph grammars) that permits the definition of correct criteria for data integrity and the consistency of configured project data model, and any dynamic changes in the project to be modeled conveniently. This approach also makes it possible to handle complex hierarchial data structures of real configured objects and to model communication and synchronization of decision making in a distributed expert system in a common formalism. The system to be configured is decomposed into structured objects, called fragments. The proposed model consists of three types of agents. D-agents are those capable of the object configuration design within the constraints on their attributes. To cope with agents' coordination, objects' consistency and data integrity problems, a special type of agent, facilitator or F-agent, is introduced. Finally, project assistants or A-agents are responsible for the user interface at the stage of object model definition. The discussion is illustrated with examples from the application domain of flexible manufacturing systems. Experimental results, current and future work on the expert system implementation are considered.     

: A distributed real-time logic language

This paper presents a new language that integrates the real-time and distributed paradigms within the framework of a concurrent logic language. Concurrent logic languages (CLLs) are capable of expressing concurrence, communication and nondeterminism in a natural way. That is, the intrinsic parallel semantics of the concurrent logic languages makes them well-suited for distributed programming. The proposed language is particularly suitable for loosely coupled systems and it contains mechanisms for distributed and real-time process control. A new execution model for concurrent logic languages is presented, which enables efficient distributed execution and real-time control. The model is introduced by giving an operational semantics for the language and the new model's implementation is discussed, including the definition of a new abstract machine and its implementation on a network of Unix workstations. Although the sequential core is not optimized, some previous results are discussed, showing the feasibility of the language's execution model for distributed real-time systems. The language is currently being used as the kernel language for a distributed simulation and validation tool for communication protocols.     

: A distributed real-time logic language

This paper presents a new language that integrates the real-time and distributed paradigms within the framework of a concurrent logic language. Concurrent logic languages (CLLs) are capable of expressing concurrence, communication and nondeterminism in a natural way. That is, the intrinsic parallel semantics of the concurrent logic languages makes them well-suited for distributed programming. The proposed language is particularly suitable for loosely coupled systems and it contains mechanisms for distributed and real-time process control. A new execution model for concurrent logic languages is presented, which enables efficient distributed execution and real-time control. The model is introduced by giving an operational semantics for the language and the new model's implementation is discussed, including the definition of a new abstract machine and its implementation on a network of Unix workstations. Although the sequential core is not optimized, some previous results are discussed, showing the feasibility of the language's execution model for distributed real-time systems. The language is currently being used as the kernel language for a distributed simulation and validation tool for communication protocols.     

Abstraction for concurrent objects

Concurrent data structures are usually designed to satisfy correctness conditions such as sequential consistency or linearizability. In this paper, we consider the following fundamental question: What guarantees are provided by these conditions for client programs? We formally show that these conditions can be characterized in terms of observational refinement. Our study also provides a new understanding of sequential consistency and linearizability in terms of abstraction of dependency between computation steps of client programs.     

Verification of concurrent control flow in distributed computer systems

An approach to verifying control flow in distributed computer systems (DCS) is presented. The approach is based on control flow checking among software components distributed over processors and cooperating among them. In this approach, control-flow behavior of DCS software is modeled and contained in special software components called verifiers. The verifiers are distributed over the processors and consulted to check the correctness of the control flow in DCS software during its execution. Algorithms for deriving the verifiers are presented. This technique can detect global errors including synchronization errors as well as local errors. It can be used for sequential or concurrent software at various levels of details. Experiments show that using this technique requires no significant overhead.<>     

Automatic geospatial metadata generation for earth science virtual data products

Recent advances in Semantic Web and Web Service technologies has shown promise for automatically deriving geospatial information and knowledge from Earth science data distributed over the Web. In a service-oriented environment, the data, information, and knowledge are often consumed or produced by complex, distributed geoscientific workflows or service chains. In order for the chaining results to be consumable, sufficient metadata for data products to be delivered by service chains must be provided. This paper proposes automatic generation of geospatial metadata for Earth science virtual data products. A virtual data product is represented using process models, and can be materialized on demand by dynamically binding and chaining archived data and services, as opposed to requiring that Earth science data products be physically archived. Semantics-enabled geospatial metadata is generated, validated, and propagated during the materialization of a virtual data product. The generated metadata not only provides a context in which end-users can interpret data products before intensive execution of service chains, but also assures semantic consistency of the service chains.     

Long-lived Rambo: Trading knowledge for communication

Shareable data services providing consistency guarantees, such as atomicity (linearizability), make building distributed systems easier. However, combining linearizability with efficiency in practical algorithms is difficult. A reconfigurable linearizable data service, called Rambo, was developed by Lynch and Shvartsman. This service guarantees consistency under dynamic conditions involving asynchrony, message loss, node crashes, and new node arrivals. The specification of the original algorithm is given at an abstract level aimed at concise presentation and formal reasoning about correctness. The algorithm propagates information by means of gossip messages. If the service is in use for a long time, the size and the number of gossip messages may grow without bound. This paper presents a consistent data service for long-lived objects that improves on Rambo in two ways: it includes an incremental communication protocol and a leave service. The new protocol takes advantage of the local knowledge, and carefully manages the size of messages by removing redundant information, while the leave service allows the nodes to leave the system gracefully. The new algorithm is formally proved correct by forward simulation using levels of abstraction. An experimental implementation of the system was developed for networks-of-workstations. The paper also includes selected analytical and preliminary empirical results that illustrate the advantages of the new algorithm.     

一种可靠可伸缩组通信系统设计与实现

组通信系统是支持一致性和容错的分布式协同系统中非常重要的组成部分.为了满足大规模协同应用的需求,文中采用了基于流言的协议与确定性协议组合的方法设计并实现了一种可靠可伸缩组通信系统SGCS.该系统主要包括可靠消息传输服务与组成员管理服务,其中基于流言的可靠多播协议和确定的消息恢复、流量控制、排序协议的组合,基于流言的失败检测协议与确定的视图一致化协议的组合以及乐观虚同步机制应用使系统具有良好的可伸缩性、可靠性和灵活性.     

Highly available and reliable communication protocols for heterogeneous systems

Existing and legacy software systems are the product of lengthy and individual development histories. Interoperability among such systems offers the support of global applications on these systems. However, interoperability among these heterogeneous systems is hampered by the absence of a reliable communication environment that supports the development of global applications.In this paper, we show how a generic communication framework can serve as a testbed for the specification, verification, and execution of distributed communication protocols. The development of distributed, global concurrency protocols is much simpler than using traditional tools, like RPC (remote procedure call), because our framework provides a high-level communication mechanism that frees the protocol designer from thinking in a message-based style. We present several protocols that are consistent with realistic assumptions about local database systems, and proofs of their correctness and consistency preservation. We also show that the execution of these protocols is fault-tolerant. The distribution of systems can be chosen according to application requirements, without adaptation of protocols. Fault tolerance can be fine-tuned within the framework itself, so that verified protocols do not need modifications in this case either.Besides protocols for traditional transaction processing, we present communication protocols for advanced transaction models that relax one or more of the ACID properties of transactions. These advanced transaction models enable local autonomy and thus are much better suited for heterogeneous environments.     

Weighted fair share scheduling for loosely controlled concurrent event systems

In asynchronous event systems, the production of an event is decoupled from its consumption via an event queue. The loose coupling of such systems allows great flexibility as to where and when within the system the event handler of a given event is actually executed, allowing them to scale with increasing number of processors on a given node or across multiple nodes in a distributed system. Although this flexibility is useful in heterogeneous distributed systems, such as SOA, it may appear not to be well suited for real-time systems, which require an upper bound on how long an event can remain unhandled in a queue. However, we show how the weighted fair scheduling algorithms developed for QoS (quality of service) routing can be adapted to event queues to bound the delay between production and consumption. We achieve this, despite the fact that the underlying execution environment is only weakly controlled, through the use of predictive techniques on a calibrated model of the event system. Our choice of algorithms and data structures is driven in part by the highly concurrent nature of the system. We show how a weighted fair scheduler can be built on a lock-free concurrent priority queue. We analyze the performance of various such queues proposed in the literature and show that a very simple linear quantizing queue yields the best performance.     

支持扩展虚拟同步的组通信算法形式描述与验证

组通信系统是为方便开发容错的分布式应用系统而提出的一种通信中间件．虚拟同步是组通信系统中的一个重要概念．其本质是限制向所有组成员递交组成员资格变化信息和应用消息的次序．为支持网络可划分的情况，引入了扩展虚拟同步模型．针对扩展虚拟同步模型的特点，提出了一种基于客户／服务器模式的组通信系统架构，并以I／O自动机的形式给出系统内部各模块的服务和算法．最后以继承建模的方式逐步给出该算法的自动机模型，并用形式化的方法验证其正确性．