Multiserver support for large-scale distributed virtual environments

CyberWalk is a distributed virtual walkthrough system that we have developed. It allows users at different geographical locations to share information and interact within a shared virtual environment (VE) via a local network or through the Internet. In this paper, we illustrate that as the number of users exploring the VE increases, the server will quickly become the bottleneck. To enable good performance, CyberWalk utilizes multiple servers and employs an adaptive region partitioning technique to dynamically partition the whole VE into regions. All objects within each region will be managed by one server. Under normal circumstances, when a viewer is exploring a region, the server of that region will be responsible for serving all requests from the viewer. When a viewer is crossing the boundary of two or more regions, the servers of all the regions involved will be serving requests from the viewer since the viewer might be able to view objects within all these regions. This is analogous to evaluating a database query using a parallel database server, which could improve the performance of serving a viewer's request tremendously. We evaluate the performance of this multiserver architecture of CyberWalk via a detail simulation model.     

Load sharing in distributed multimedia-on-demand systems

Service providers have begun to offer multimedia-on-demand services to residential estates by installing isolated, small-scale multimedia servers at individual estates. Such an arrangement allows the service providers to operate without relying on a highspeed, large-capacity metropolitan area network, which is still not available in many countries. Unfortunately, installing isolated servers can incur very high server costs, as each server requires spare bandwidth to cope with fluctuations in user demand. The authors explore the feasibility of linking up several small multimedia servers to a (limited-capacity) network, and allowing servers with idle retrieval bandwidth to help out servers that are temporarily overloaded; the goal is to minimize the waiting time for service to begin. We identify four characteristics of load sharing in a distributed multimedia system that differentiate it from load balancing in a conventional distributed system. We then introduce a GWQ load sharing algorithm that fits and exploits these characteristics; it puts all servers' pending requests in a global queue, from which a server with idle capacity obtains additional jobs. The performance of the algorithm is captured by an analytical model, which we validate through simulations. Both the analytical and simulation models show that the algorithm vastly reduces wait times at the servers. The analytical model also provides guidelines for capacity planning. Finally, we propose an enhanced GWQ+L algorithm that allows a server to reclaim active local requests that are being serviced remotely. Simulation experiments indicate that the scheduling decisions of GWQ+L are optimal, i.e., it enables the distributed servers to approximate the performance of a large centralized server     

Independent recovery in large-scale distributed systems

In large systems, replication can become important means to improve data access times and availability. Existing recovery protocols, on the other hand, were proposed for small-scale distributed systems. Such protocols typically update stale, newly-recovered sites with replicated data and resolve the commit uncertainty of recovering sites. Thus, given that in large systems failures are more frequent and that data access times are costlier, such protocols can potentially introduce large overheads in large systems and must be avoided, if possible. We call these protocols dependent recovery protocols since they require a recovering site to consult with other sites. Independent recovery has been studied in the context of one-copy systems and has been proven unattainable. This paper offers independent recovery protocols for large-scale systems with replicated data. It shows how the protocols can be incorporated into several well-known replication protocols and proves that these protocols continue to ensure data consistency. The paper then addresses the issue of nonblocking atomic commitment. It presents mechanisms which can reduce the overhead of termination protocols and the probability of blocking. Finally, the performance impact of the proposed recovery protocols is studied through the use of simulation and analytical studies. The results of these studies show that the significant benefits of independent recovery can be enjoyed with a very small loss in data availability and a very small increase in the number of transaction abortions     

Regeneration with virtual copies for distributed computing systems

The authors consider the consistency control problem for replicated data in a distributed computing system (DCS) and propose a new algorithm to dynamically regenerate copies of data objects in response to node failures and network partitioning in the system. The DCS is assumed to have strict consistency constraints for data object copies. The algorithm combines the advantages of voting-based algorithms and regeneration mechanisms to maintain mutual consistency of replicated data objects in the case of node failures and network partitioning. The algorithm extends the feasibility of regeneration to DCS on wide area networks and is able to satisfy user queries as long as there is one current partition in the system. A stochastic availability analysis of the algorithm shows that it provides improved availability as compared to previously proposed dynamic voting algorithms     

Load sharing in distributed systems with failures

Summary An approximate model is presented for the mean response time in a distributed computer system in which components may fail. Each node in the system periodically performs a checkpoint, and also periodically tests the other nodes to determine whether they are failed or not. When a node fails, it distributes its workload to other nodes which appear to be operational, based on the results of its most recent test. An approximate response time model is developed, explicitly allowing for the delays caused by transactions being incorrectly transferred to failed nodes, because of out-of-date testing results. For the case when all nodes are identical, a closed form solution is derived for the optimal testing rate minimizing the average response time. Numerical results are presented illustrating the relationships among the problem parameters.This research was performed while Satish Tripathi and David Finkel were visiting ISEM. Satish Tripathi's research was supported in part by grants from NSF (grant no. DCR-84-05235) and NASA (grant no. NAG 5-235), and by Université de Paris-Sud     

Adaptive load sharing in heterogeneous distributed systems

In this paper, we study the performance characteristics of simple load sharing algorithms for heterogeneous distributed systems. We assume that nonnegligible delays are encountered in transferring jobs from one node to another. We analyze the effects of these delays on the performance of two threshold-based algorithms called Forward and Reverse. We formulate queuing theoretic models for each of the algorithms operating in heterogeneous systems under the assumption that the job arrival process at each node in Poisson and the service times and job transfer times are exponentially distributed. The models are solved using the Matrix-Geometric solution technique. These models are used to study the effects of different parameters and algorithm variations on the mean job response time: e.g., the effects of varying the thresholds, the impact of changing the probe limit, the impact of biasing the probing, and the optimal response times over a large range of loads and delays. Wherever relevant, the results of the models are compared with the M/M/ 1 model, representing no load balancing (hereafter referred to as NLB), and the M/M/K model, which is an achievable lower bound (hereafter referred to as LB).     

Risk modeling in distributed, large-scale systems

Risk is inherent in distributed, large-scale systems. The paper explores the challenges of risk modeling in such systems, and suggests a risk modeling approach that is responsive to the requirements of complex, distributed, large-scale systems. An example of the use of the approach in the marine transportation system is given. The paper concludes with a discussion of limitations of the approach and of future work     

A macroeconomic model for resource allocation in large-scale distributed systems

In this paper we discuss an economic model for resource sharing in large-scale distributed systems. The model captures traditional concepts such as consumer satisfaction and provider revenues and enables us to analyze the effect of different pricing strategies upon measures of performance important for the consumers and the providers. We show that given a particular set of model parameters the satisfaction reaches an optimum; this value represents the perfect balance between the utility and the price paid for resources. Our results confirm that brokers play a very important role and can influence positively the market. We also show that consumer satisfaction does not track the consumer utility; these two important performance measures for consumers behave differently under different pricing strategies. Pricing strategies also affect the revenues obtained by providers, as well as, the ability to satisfy a larger population of users.     

Improving the performance of distributed virtual environment systems

The last years have witnessed a dramatic growth in the number as well as in the variety of distributed virtual environment systems. These systems allow multiple users, working on different client computers that are interconnected through different networks, to interact in a shared virtual world. One of the key issues in the design of scalable and cost-effective DVE systems is the partitioning problem. This problem consists of efficiently assigning the existing clients to the servers in the system and some techniques have been already proposed for solving it. This paper experimentally analyzes the correlation of the quality function proposed in the literature for solving the partitioning problem with the performance of DVE systems. Since the results show an absence of correlation, we also propose the experimental characterization of DVE systems. The results show that the reason for that absence of correlation is the nonlinear behavior of DVE systems with regard to the number of clients in the system. DVE systems reach saturation when any of the servers reaches 100 percent of CPU utilization. The system performance greatly decreases if this limit is exceeded in any server. Also, as a direct application of these results, we present a partitioning method that is targeted to keep all the servers in the system below a certain threshold value of CPU utilization, regardless of the amount of network traffic. Evaluation results show that the proposed partitioning method can improve DVE system performance, regardless of both the movement pattern of clients and the initial distribution of clients in the virtual world.     

Model-based testing of global properties on large-scale distributed systems

ContextLarge-scale distributed systems are becoming commonplace with the large popularity of peer-to-peer and cloud computing. The increasing importance of these systems contrasts with the lack of integrated solutions to build trustworthy software. A key concern of any large-scale distributed system is the validation of global properties, which cannot be evaluated on a single node. Thus, it is necessary to gather data from distributed nodes and to aggregate these data into a global view. This turns out to be very challenging because of the system’s dynamism that imposes very frequent changes in local values that affect global properties. This implies that the global view has to be frequently updated to ensure an accurate validation of global properties.ObjectiveIn this paper, we present a model-based approach to define a dynamic oracle for checking global properties. Our objective is to abstract relevant aspects of such systems into models. These models are updated at runtime, by monitoring the corresponding distributed system.MethodWe conduce real-scale experimental validation to evaluate the ability of our approach to check global properties. In this validation, we apply our approach to test two open-source implementations of distributed hash tables. The experiments are deployed on two clusters of 32 nodes.ResultsThe experiments reveal an important defect on one implementation and show clear performance differences between the two implementations. The defect would not be detected without a global view of the system.ConclusionTesting global properties on distributed software consists of gathering data from different nodes and building a global view of the system, where properties are validated. This process requires a distributed test architecture and tools for representing and validating global properties. Model-based techniques are an expressive mean for building oracles that validate global properties on distributed systems.     

An Adaptive Dynamic Grid-Based approach to DDM for large-scale distributed simulation systems

Data Distribution Management (DDM) plays a key role in traffic volume control of large-scale distributed simulations. In recent years, several solutions have been devised to make DDM more efficient and adaptive to different traffic conditions. Examples of such systems include the Region-Based, Fixed Grid-Based, Hybrid, and Dynamic Grid-Based (DGB) schemes. However, less effort has been directed toward improving the processing performance of DDM techniques. This paper presents a novel DDM scheme called the Adaptive Dynamic Grid-Based (ADGB) scheme that optimizes DDM time through analysis of matching performance. ADGB uses an advertising scheme in which information about the target cell involved in the process of matching subscribers to publishers is known in advance. An important concept known as the Distribution Rate (DR) is devised. The distribution rate represents the relative processing load and communication load generated at each federate. The matching performance and the distribution rate are used as part of the ADGB method to select, throughout the simulation, the devised advertisement scheme that achieves the maximum gain with acceptable network traffic overhead. If we assume the same worst case propagation delays, when the matching probability is high, the performance estimation of ADGB has shown that a maximum efficiency gain of 66% can be achieved over the Dynamic Grid-Based scheme. The novelty of the ADGB scheme is its focus on improving performance, an important (and often forgotten) goal of DDM strategies.     

A survey of recoverable distributed shared virtual memory systems

Distributed Shared Virtual Memory (DSVM) systems provide a shared memory abstraction on distributed memory architectures. Such systems ease parallel application programming because the shared-memory programming model is often more natural than the message-passing paradigm. However, the probability of failure of a DSVM increases with the number of sites. Thus, fault tolerance mechanisms must be implemented in order to allow processes to continue their execution in the event of a failure. This paper gives an overview of recoverable DSVMs (RDSVMs) that provide a checkpointing mechanism to restart parallel computations in the event of a site failure     

Utopia: A load sharing facility for large,heterogeneous distributed computer systems

Load sharing in large, heterogeneous distributed systems allows users to access vast amounts of computing resources scattered around the system and may provide substantial performance improvements to applications. We discuss the design and implementation issues in Utopia, a load sharing facility specifically built for large and heterogeneous systems. The system has no restriction on the types of tasks that can be remotely executed, involves few application changes and no operating system change, supports a high degree of transparency for remote task execution, and incurs low overhead. The algorithms for managing resource load information and task placement take advantage of the clustering nature of large-scale distributed systems; centralized algorithms are used within host clusters, and directed graph algorithms are used among the clusters to make Utopia scalable to thousands of hosts. Task placements in Utopia exploit the heterogeneous hosts and consider varying resource demands of the tasks. A range of mechanisms for remote execution is available in Utopia that provides varying degrees of transparency and efficiency. A number of applications have been developed for Utopia, ranging from a load sharing command interpreter, to parallel and distributed applications, to a distributed batch facility. For example, an enhanced Unix command interpreter allows arbitrary commands and user jobs to be executed remotely, and a parallel make facility achieves speed-ups of 15 or more by processing a collection of tasks in parallel on a number of hosts.     

Schema mediation for large-scale semantic data sharing

Intuitively, data management and data integration tools should be well suited for exchanging information in a semantically meaningful way. Unfortunately, they suffer from two significant problems: they typically require a common and comprehensive schema design before they can be used to store or share information, and they are difficult to extend because schema evolution is heavyweight and may break backward compatibility. As a result, many large-scale data sharing tasks are more easily facilitated by non-database-oriented tools that have little support for semantics.The goal of the peer data management system (PDMS) is to address this need: we propose the use of a decentralized, easily extensible data management architecture in which any user can contribute new data, schema information, or even mappings between other peers schemas. PDMSs represent a natural step beyond data integration systems, replacing their single logical schema with an interlinked collection of semantic mappings between peers individual schemas.This paper considers the problem of schema mediation in a PDMS. Our first contribution is a flexible language for mediating between peer schemas that extends known data integration formalisms to our more complex architecture. We precisely characterize the complexity of query answering for our language. Next, we describe a reformulation algorithm for our language that generalizes both global-as-view and local-as-view query answering algorithms. Then we describe several methods for optimizing the reformulation algorithm and an initial set of experiments studying its performance. Finally, we define and consider several global problems in managing semantic mappings in a PDMS.Received: 16 December 2002, Accepted: 14 April 2003, Published online: 12 December 2003Edited by: V. Atluri     

A distributed admission control model for QoS assurance in large-scale media delivery systems

Conventional admission control models incur some performance penalty. First, admission control computation can overload a server that is already heavily loaded. Also, in large-scale media systems with geographically distributed server clusters, performing admission control on each cluster can result in long response latency, if the client request is denied at one site and has to be forwarded to another site. Furthermore, in prefix caching, initial frames cached at the proxy are delivered to the client before the admission decisions are made. If the media server is heavily loaded and, finally, has to deny the client request, forwarding a large number of initial frames is a waste of critical network resources. In this paper, a novel distributed admission control model is presented. We make use of proxy servers to perform the admission control tasks. Each proxy hosts an agent to coordinate the effort. Agents reserve media server's disk bandwidth and make admission decisions autonomously based on the allocated disk bandwidth. We develop an effective game theoretic framework to achieve fairness in the bandwidth allocation among the agents. To improve the overall bandwidth utilization, we also consider an aggressive admission control policy where each agent may admit more requests than its allocated bandwidth allows. The distributed admission control approach provides the solution to the stated problems incurred in conventional admission control models. Experimental studies show that our algorithms significantly reduce the response latency and the media server load.     

Systems analysis of life cycle of large-scale information-control systems

Results of analysis of problems associated with life cycle cost reduction for large-scale information-control systems are presented. Rational design concepts with application of an aspect-oriented method for large-scale systems are described. An approach to life cycle organization corresponding to the ISO/IEC 12207-2008 standard on the basis of domain engineering and model-driven engineering technologies is proposed. The paper shows a necessity of using the approach within a unified mathematical semantic base supplied by category theory. The approach has been tested during development of application information-control systems in energy production.     

分布式资源共享的网格信息服务

网格技术使得广泛的大规模的资源共享成为可能。在如此分布、多样、动态的网格环境中,信息服务成为任何网格软件框架的重要部分,它为发现和监控提供了最根本的机制,也因此为系统规划应用行为奠定了基础。介绍的信息服务系统结构,明确了底层查询和注册协议以及优化方法,从而简了化单个实体纳入（包括支持不同查询语言和发现策略的集合索引）信息结构的过程。