Scalable dimensioning of resilient Lambda Grids

Grids consist of the aggregation of numerous dispersed computational, storage and network resources, able to satisfy even the most demanding computing jobs. Due to the data-intensive nature of Grid jobs, there is an increasing interest in Grids using optical transport networks as this technology allows for the timely delivery of large amounts of data. Such Grids are commonly referred to as Lambda Grids.

An important aspect of Grid deployment is the allocation and activation of installed network capacity, needed to transfer data and jobs to and from remote resources. However, the exact nature of a Grid’s network traffic depends on the way arriving workload is scheduled over the various Grid sites. As Grids possibly feature high numbers of resources, jobs and users, solving the combined Grid network dimensioning and workload scheduling problem requires the use of scalable mathematical methods such as Divisible Load Theory (DLT). Lambda Grids feature additional complexity such as wavelength granularity and continuity or conversion constraints must be enforced. Additionally, Grid resources cannot be expected to be available at all times. Therefore, the extra complexity of resilience against possible resource failures must be taken into account when modelling the combined Grid network dimensioning and workload scheduling problem, enforcing the need for scalable solution methods. In this work, we tackle the Lambda Grid combined dimensioning and workload scheduling problem and incorporate single-resource failure or unavailability scenarios. We use Divisible Load Theory to tackle the scalability problem and compare non-resilient lambda Grid dimensioning to the dimensions needed to survive single-resource failures. We distinguish three failure scenarios relevant to lambda Grid deployment: computational element, network link and optical cross-connect failure. Using regular network topologies, we derive analytical bounds on the dimensioning cost. To validate these bounds, we present comparisons for the resulting Grid dimensions assuming a 2-tier Grid operation as a function of varying wavelength granularity, fiber/wavelength cost models, traffic demand asymmetry and Grid scheduling strategy for a specific set of optical transport networks.     

Workload dynamics on clusters and grids

This paper presents a comprehensive statistical analysis of a variety of workloads collected on production clusters and Grids. The applications are mostly computational-intensive and each task requires single CPU for processing data, which dominate the workloads on current production Grid systems. Trace data obtained on a parallel supercomputer is also included for comparison studies. The statistical properties of workloads are investigated at different levels, including the Virtual Organization (VO) and user behavior. The aggregation procedure and scaling analysis are applied to job arrivals, leading to the identifications of several basic patterns, namely pseudo-periodicity, long range dependence (LRD), and multifractals. It is shown that statistical measures based on interarrivals are of limited usefulness and count based measures should be trusted when it comes to correlations. Other job characteristics like run time and memory consumption are also studied. A “bag-of-tasks” behavior is empirically evidenced, strongly indicating temporal locality. The nature of such dynamics in the Grid workloads is discussed. This study has important implications on workload modeling and performance predictions, and points out the need of comprehensive performance evaluation studies given the workload characteristics.

An Energy-Efficient Resource Allocation Scheme for Mobile Ad Hoc Computational Grids

Due to recent advancements in mobile computing and communication technologies, mobile ad hoc computational Grids are emerging as a new computing paradigm, enabling innovative applications through sharing of computing resources among mobile devices without any pre-existing network infrastructure. Energy-efficient resource allocation is one of the key issues in mobile ad hoc computational Grids due to limited battery life of mobile nodes. To reduce energy consumption, we propose a hybrid power-based resource allocation scheme for allocation of interdependent tasks to nodes within mobile ad hoc computational Grid. The basic idea is to exploit dependencies and task type, and allocate interdependent tasks to nodes accessible at minimum transmission power. We also propose a power-based algorithm to search a group of closest nodes to allocate a set of interdependent tasks. Compared to traditional algorithms, complexity of proposed algorithm depends on number of transmission power levels rather than number of nodes within a Grid. The scheme is validated in a simulation environment using various workloads and parameters.     

Market-oriented Grids and Utility Computing: The State-of-the-art and Future Directions

Traditional resource management techniques (resource allocation, admission control and scheduling) have been found to be inadequate for many shared Grid and distributed systems, that consist of autonomous and dynamic distributed resources contributed by multiple organisations. They provide no incentive for users to request resources judiciously and appropriately, and do not accurately capture the true value, importance and deadline (the utility) of a user’s job. Furthermore, they provide no compensation for resource providers to contribute their computing resources to shared Grids, as traditional approaches have a user-centric focus on maximising throughput and minimising waiting time rather than maximising a providers own benefit. Consequently, researchers and practitioners have been examining the appropriateness of ‘market-inspired’ resource management techniques to address these limitations. Such techniques aim to smooth out access patterns and reduce the chance of transient overload, by providing a framework for users to be truthful about their resource requirements and job deadlines, and offering incentives for service providers to prioritise urgent, high utility jobs over low utility jobs. We examine the recent innovations in these systems (from 2000–2007), looking at the state-of-the-art in price setting and negotiation, Grid economy management and utility-driven scheduling and resource allocation, and identify the advantages and limitations of these systems. We then look to the future of these systems, examining the emerging ‘Catallaxy’ market paradigm. Finally we consider the future directions that need to be pursued to address the limitations of the current generation of market oriented Grids and Utility Computing systems.     

Resource brokering using a multi‐site resource allocation strategy for computational grids

Grid computing employs heterogeneous resources which may be installed on different platforms, hardware/software, computer architectures, and perhaps using different computer languages to solve large‐scale computational problems. As many more Grids are being developed worldwide, the number of multi‐institutional collaborations is growing rapidly. However, to realize Grid computing's full potential, it is expected that Grid participants must be able to share one another's resources. This paper presents a resource broker that employs the multi‐site resource allocation (MSRA) strategy and the dynamic domain‐based network information model that we propose to allocate Grid resources to submitted jobs, where the Grid resources may be dispersed at different sites, and owned and governed by different organizations or institutes. The jobs and resources may also belong to different clusters/sites. Resource statuses collected by the Ganglia, and network bandwidths gathered by the Network Weather Service, are both considered in the proposed scheduling approach. A dynamic domain‐based model for network information measurement is also proposed to choose the most appropriate resources that meet the jobs' execution requirements. Experimental results show that MSRA outperformed the other tested strategies. Copyright © 2010 John Wiley & Sons, Ltd.     

A Market-Oriented Grid Directory Service for Publication and Discovery of Grid Service Providers and their Services

The emergence of Grids as a platform for sharing and aggregation of distributed resources increases the need for mechanisms that allow an efficient management of resources. The Grid economy has been identified as one of the potential solutions as it helps in managing the supply and demand for resources and enables sustained sharing of resources by providing economic incentive for Grid resource providers. An economy based Grid computing environment needs to support an infrastructure that enables the creation of a marketplace for meeting of providers and consumers. This paper presents the Grid Market Directory (GMD) that serves as a registry for publication and discovery of Grid service providers and their services.     

Multi-criteria and satisfaction oriented scheduling for hybrid distributed computing infrastructures

Assembling and simultaneously using different types of distributed computing infrastructures (DCI) like Grids and Clouds is an increasingly common situation. Because infrastructures are characterized by different attributes such as price, performance, trust, and greenness, the task scheduling problem becomes more complex and challenging. In this paper we present the design for a fault-tolerant and trust-aware scheduler, which allows to execute Bag-of-Tasks applications on elastic and hybrid DCI, following user-defined scheduling strategies. Our approach, named Promethee scheduler, combines a pull-based scheduler with multi-criteria Promethee decision making algorithm. Because multi-criteria scheduling leads to the multiplication of the possible scheduling strategies, we propose SOFT, a methodology that allows to find the optimal scheduling strategies given a set of application requirements. The validation of this method is performed with a simulator that fully implements the Promethee scheduler and recreates an hybrid DCI environment including Internet Desktop Grid, Cloud and Best Effort Grid based on real failure traces. A set of experiments shows that the Promethee scheduler is able to maximize user satisfaction expressed accordingly to three distinct criteria: price, expected completion time and trust, while maximizing the infrastructure useful employment from the resources owner point of view. Finally, we present an optimization which bounds the computation time of the Promethee algorithm, making realistic the possible integration of the scheduler to a wide range of resource management software.     

基于动态备份的容错网格任务调度*

提出了一种基于自适应备份的网格容错任务调度算法:最高百分之k备份算法.该算法对任务的安全需求和资源的信任等级进行匹配,在系统安全等级较低并且网络和主机可能失效的网格环境中进行容错任务调度.调度时,该算法根据整个网格系统的安全状况,对具有最高安全需求的百分之k的任务进行动态备份,任务备份数根据系统安全状况自适应变化,并对失败的任务重新调度.仿真结果表明,该算法可以有效提高不安全网格环境下的任务调度成功率,具有很好的容错性和可扩展性,优于固定备份数的网格任务调度算法.     

Performance analysis of allocation policies for interGrid resource provisioning

Several Grids have been established and used for varying science applications during the last years. Most of these Grids, however, work in isolation and with different utilisation levels. Previous work has introduced an architecture and a mechanism to enable resource sharing amongst Grids. It has demonstrated that there can be benefits for a Grid to offload requests or provide spare resources to another Grid. In this work, we address the problem of resource provisioning to Grid applications in multiple-Grid environments. The provisioning is carried out based on availability information obtained from queueing-based resource management systems deployed at the provider sites which are the participants of the Grids. We evaluate the performance of different allocation policies. In contrast to existing work on load sharing across Grids, the policies described here take into account the local load of resource providers, imprecise availability information and the compensation of providers for the resources offered to the Grid. In addition, we evaluate these policies along with a mechanism that allows resource sharing amongst Grids. Experimental results obtained through simulation show that the mechanism and policies are effective in redirecting requests thus improving the applications’ average weighted response time.     

Assessing Green Strategies in Peer-to-Peer Opportunistic Grids

Opportunistic peer-to-peer (P2P) Grids are distributed computing infrastructures that harvest the idle computing cycles of computing resources geographically distributed. In these Grids, the demand for resources is typically bursty. During bursts of resource demand, many Grid resources are required, but on other occasions they may remain idle for long periods of time. If the resources are kept powered on even when they are neither processing their owners’ workload nor Grid jobs, their exploitation is not efficient in terms of energy consumption. One way to reduce the energy consumed in these idleness periods is to place the computers that form the Grid in a “sleeping” state which consumes less energy. In Grid computing, this strategy introduces a tradeoff between the benefit of energy saving and the associated costs in terms of increasing the job response time, also known as makespan, and reducing the hard disks’ lifetime. To mitigate these costs, it is usually introduced a timeout policy together with the sleeping state, which tries to avoid useless state transitions. In this work, we use simulations to analyze the potential of using sleeping states to save energy in each site of a P2P Grid. Our results show that sleeping states can save energy with low associated impact on jobs’ makespan and hard disks’ lifetime. Furthermore, the best sleeping strategy to be used depends on the characteristics of each individual site, thus, each site should be configured to use the sleeping strategy that best fits its characteristics. Finally, differently from other kinds of Grid infrastructures, P2P Grids can place a machine in sleeping mode as soon as it becomes idle, i.e. it is not necessary to use an aggressive timeout policy. This allows increases on the Grid’s energy saving without impacting significantly the jobs’ makespan and the disks’ lifetime.