A taxonomy of scheduling in general-purpose distributed computingsystems

One measure of the usefulness of a general-purpose distributed computing system is the system's ability to provide a level of performance commensurate to the degree of multiplicity of resources present in the system. A taxonomy of approaches to the resource management problem is presented in an attempt to provide a common terminology and classification mechanism necessary in addressing this problem. The taxonomy, while presented and discussed in terms of distributed scheduling, is also applicable to most types of resource management     

Evolving to distributed database environments

As the computer industry begins to experience a transition from centralized processors to distributed environments of networked processors, a methodology for conversion must be considered. Various considerations are identified in this paper that affect the development of this new technology. The impact of data models for distributed databases is considered. The concept of a dictory, a dictionary/directory facility, is developed as the focal point for extension into the distributed environment.     

Security monitoring in heterogeneous globally distributed environments

Keeping control of a highly heterogeneous network of distributed systems in any global company is a difficult task. This problem cannot be brought under control with any single approach or technology. We must focus on the objectives and build tools into a common integrated framework to provide a security state and event monitoring concept for the entire organisation.     

Clustering distributed data streams in peer-to-peer environments

This paper describes a technique for clustering homogeneously distributed data in a peer-to-peer environment like sensor networks. The proposed technique is based on the principles of the K-Means algorithm. It works in a localized asynchronous manner by communicating with the neighboring nodes. The paper offers extensive theoretical analysis of the algorithm that bounds the error in the distributed clustering process compared to the centralized approach that requires downloading all the observed data to a single site. Experimental results show that, in contrast to the case when all the data is transmitted to a central location for application of the conventional clustering algorithm, the communication cost (an important consideration in sensor networks which are typically equipped with limited battery power) of the proposed approach is significantly smaller. At the same time, the accuracy of the obtained centroids is high and the number of samples which are incorrectly labeled is also small.     

Virtual distributed environments in a shared infrastructure

We have developed a middleware system that integrates and extends virtual machine and virtual network technologies to support mutually isolated virtual distributed environments in shared infrastructures like the grid and the PlanetLab overlay infrastructure. Integrating virtual network and on-demand virtual machine creation and customization technologies makes virtual distributed environments a reality. The Violin-based middleware system integrates and enhances such technologies to create virtual distributed environments.     

Improved QMRCGSTAB method in distributed parallel environments

An improved version of the QMRCGSTAB method (IQMRCGSTAB method) for solving large sparse linear systems with non-symmetric coefficient matrices is proposed. The proposed method combines elements of numerical stability and parallel algorithm design without increasing computational costs. Performance analysis shows that the IQMRCGSTAB method has better parallelism and scalability than the original method. Numerical experiments show the efficiency of the new method.     

Rapid prototyping for distributed virtual environments

The authors combine rapid and exploratory prototyping techniques to capture initially unknown, interdependent requirements in the development of distributed virtual environment applications. They employ containerization to accelerate data exchange     

Analysis of replication in distributed database systems

The authors develop an approximate analytical model to study the tradeoffs of replicating data in a distributed database environment. Several concurrency control protocols are considered, including pessimistic, optimistic, and semi-optimistic protocols. The approximate analysis captures the effect of the protocol on hardware resource contention and data contention. The accuracy of the approximation is validated through detailed simulations. It is found that the benefit of replicating data and the optimal number of replicates are sensitive to the concurrency control protocol. Under the optimistic and semi-optimistic protocols, replications can significantly improve response time with an additional MIPS (million instructions per second) requirement to maintain consistency among the replicates. The optimal degree of replication is further affected by the transaction mix (e.g. the fraction of read-only transactions), the communications delay and overhead, the number of distributed sites, and the available MIPS. Sensitivity analyses have been carried out to examine how the optimal degree of replication changes with respect to these factors     

Analysis of workflow schedulers in simulated distributed environments

The Journal of Supercomputing - Task graphs provide a simple way to describe scientific workflows (sets of tasks with dependencies) that can be executed on both HPC clusters and in the cloud. An...     

Reliable distributed data stream management in mobile environments

The proliferation of sensor technology, especially in the context of embedded systems, has brought forward novel types of applications that make use of streams of continuously generated sensor data. Many applications like telemonitoring in healthcare or roadside traffic monitoring and control particularly require data stream management (DSM) to be provided in a distributed, yet reliable way. This is even more important when DSM applications are deployed in a failure-prone distributed setting including resource-limited mobile devices, for instance in applications which aim at remotely monitoring mobile patients. In this paper, we introduce a model for distributed and reliable DSM. The contribution of this paper is threefold. First, in analogy to the SQL isolation levels, we define levels of reliability and describe necessary consistency constraints for distributed DSM that specify the tolerated loss, delay, or re-ordering of data stream elements, respectively. Second, we use this model to design and analyze an algorithm for reliable distributed DSM, namely efficient coordinated operator checkpointing (ECOC). We show that ECOC provides lossless and delay-limited reliable data stream management and thus can be used in critical application domains such as healthcare, where the loss of data stream elements cannot be tolerated. Third, we present detailed performance evaluations of the ECOC algorithm running on mobile, resource-limited devices. In particular, we can show that ECOC provides a high level of reliability while, at the same time, featuring good performance characteristics with moderate resource consumption.     

Resource failures risk assessment modelling in distributed environments

Service providers offer access to resources and services in distributed environments such as Grids and Clouds through formal Service level Agreements (SLA), and need well-balanced infrastructures so that they can maximise the Quality of Service (QoS) they offer and minimise the number of SLA violations. We propose a mathematical model to predict the risk of failure of resources in such environments using a discrete-time analytical model driven by reliability functions fitted to observed data. The model relies on the resource historical data so as to predict the risk of failure for a given time interval. The model is evaluated by comparing the predicted risk of failure with the observed risk of failure, and is shown to accurately predict the resources risk of failure, allowing a service provider to selectively choose which SLA request to accept.     

Making filters smart in distributed data stream environments

Monitoring aggregate queries in real-time over distributed streaming environments appears to be a great challenge not only because of the huge data volume and high rate, but also because of the limitation of the network transmission bandwidth. Consequently, ensuring qualified approximate results with economical network consumption becomes one of the most important goals in such scenarios. In this paper, we study how to monitor aggregate queries continuously over distributed environments efficiently by disposing numerous filters at remote sites, in order to transmit only a small part of incoming data to the query site and therefore save the network resource significantly. We also show how to adjust the parameters of a filter continuously when the incoming data distribution at the corresponding remote site changes. Analysis and extensive experimental results demonstrate that our approach outperforms the existing work.     

Direction-aware resource discovery in large-scale distributed computing environments

As a system scales up, the peer-to-peer (P2P) approach is attractive to distributed computing environments, such as Grids and Clouds, due to the amount of resources increased. The major issue in large-scale distributed systems is to prevent the phenomenon of a communication bottleneck or a single point of failure. Conventional approaches may not be able to apply directly to such environments due to restricted queries and varied resource characteristics. Alternatively, a fully decentralized resource discovery service based on an unstructured overlay, which relies only on the information of resource attributes and characteristics, may be a feasible solution. One major challenge of such service is to locate desired and suitable resources without the global knowledge of distributed sharing resources. As a consequence, the more nodes the resource discovery service involves, the higher the network overhead incurs. In this paper, we proposed a direction-aware strategy which can alleviate the network traffic among unstructured information systems for distributed resource discovery service. Experimental results have demonstrated that the proposed approach achieves higher success rate at low cost and higher scalability.     

Job control in distributed environments with non-dedicated resources

A model of job processing in distributed environments with non-dedicated resources having owners is proposed and justified. The model adopts the implementation of the so-called fair distribution of resources between independent users and owners of computation nodes. Problems of optimal and efficient combinations of available resources based on dynamically changing schedules (lists of slots of which each is associated with a time interval for the execution of a part of job and a certain type of resources) are discussed.     

ViWoSG: A distributed scene graph of ultramassive distributed virtual environments

An ultra-massive distributed virtual environment generally consists of ultra-massive terrain data and a large quantity of objects and their attribute data, such as 2D/3D geometric models, audio/video, images, vectors, characteristics, etc. In this paper, we propose a novel method for constructing distributed scene graphs with high extensibility. Thismethod can support high concurrent interaction of clients and implement various tasks such as editing, querying, accessing and motion controlling. Some application experiments are performed to demonstrate its efficiency and soundness. Supported by the National Basic Research Program of China (Grant No. 2004CB719403), the National High-Tech Research & Development Program of China (Grant Nos. 2006AA01Z334, 2007AA01Z318, 2009AA01Z324), the National Natural Science Foundation of China (Grant Nos. 60573151, 60703062, 60833007), and the Marine 908-03-01-10 Project     

Coding-based replication schemes for distributed systems

Data is often replicated in distributed systems to improve availability and performance. This replication is expensive in terms of disk storage since the existing schemes generally require full files to be stored at each site. In this paper, we present schemes which significantly reduce the storage requirements in replication based systems. These schemes use the coding method suggested by Rabin to store replicated data. The first scheme that we present is a modification of the simple voting algorithm and its quorum requirements. We then show how some of the extensions of the voting algorithm can also be modified to get storage efficient schemes for managing such replication. We evaluate the availability offered by these schemes and show that the storage space required to achieve certain availability are significantly lower than the conventional schemes with full file replication. Since coding is used, these schemes also provide a high degree of data security     

Fully distributed three-tier active software replication

Keeping strongly consistent the state of the replicas of a software service deployed across a distributed system prone to crashes and with highly unstable message transfer delays (e.g., the Internet), is a real practical challenge. The solution to this problem is subject to the FLP impossibility result, and thus there is a need for "long enough" periods of synchrony with time bounds on process speeds and message transfer delays to ensure deterministic termination of any run of agreement protocols executed by replicas. This behavior can be abstracted by a partially synchronous computational model. In this setting, before reaching a period of synchrony, the underlying network can arbitrarily delay messages and these delays can be perceived as false failures by some timeout-based failure detection mechanism leading to unexpected service unavailability. This paper proposes a fully distributed solution for active software replication based on a three-tier software architecture well-suited to such a difficult setting. The formal correctness of the solution is proved by assuming the middle-tier runs in a partially synchronous distributed system. This architecture separates the ordering of the requests coming from clients, executed by the middle-tier, from their actual execution, done by replicas, i.e., the end-tier. In this way, clients can show up in any part of the distributed system and replica placement is simplified, since only the middle-tier has to be deployed on a well-behaving part of the distributed system that frequently respects synchrony bounds. This deployment permits a rapid timeout tuning reducing thus unexpected service unavailability.