A secure information service for monitoring large scale grids

The Grid Information Service (GIS) is a core component in the Grid software infrastructure. It provides diverse information to users or other service components in Grid environments. In this paper, we propose a scalable GIS architecture for information management in a large scale Grid Virtual Organization (VO). This architecture consists of the VO layer, site layer and resource layer: at the resource layer, information agents and pluggable information sensors are deployed on each resource monitored. This information agent and sensor approach provides a flexible framework that enables specific information to be captured; at the site layer, a site information service component with caching capability aggregates and maintains up-to-date information of all the resources monitored within an administrative domain; at the VO layer, a peer-to-peer approach is used to build a virtual network of site information services for information discovery and query in a large scale Grid VO. This decentralized approach makes information management scalable and robust. Furthermore, we propose a security framework for the GIS, which provide security policies for authentication and authorization control of the GIS at both the site and the VO layers. Our GIS has been implemented based on the Globus Toolkit 4 as Web services compliant to Web Services Resource Framework (WSRF) specifications. The experimental results show that the GIS presents satisfactory scalability in handling information for large scale Grids.     

Dynamic Instrumentation, Performance Monitoring and Analysis of Grid Scientific Workflows

While existing work concentrates on developing QoS models of business workflows and Web services, few tools have been developed to support the monitoring and performance analysis of scientific workflows in Grids. This paper describes novel Grid services for dynamic instrumentation of Grid-based applications, performance monitoring and analysis of Grid scientific workflows. We describe a Grid dynamic instrumentation service that provides a widely accessible interface for other services and users to conduct the dynamic instrumentation of Grid applications during the runtime. We introduce a Grid performance analysis service for Grid scientific workflows. The analysis service utilizes various types of data including workflow graphs, monitoring data of resources, execution status of activities, and performance measurements obtained from the dynamic instrumentation of invoked applications, and provides a rich set of functionalities and features to support the online monitoring and performance analysis of scientific workflows. Workflows and their relevant information including performance metrics are stored and utilized for comparing the performance of constructs of different workflows and for supporting multi-workflow analysis. The work described in this paper is supported in part by the Austrian Science Fund as part of the Aurora Project under contract SFBF1104 and by the European Union through the IST-2002-511385 project K-WfGrid.     

Coordinated harnessing of the IRISGrid and EGEE testbeds with GridWay

Since the late 1990s, we have witnessed an extraordinary development of Grid technologies. Nowadays, different Grids are being deployed within the context of a growing number of national and transnational research projects. However, the coexistence of those different infrastructures involves two challenging issues, namely: (i) simultaneous and coordinated use of resources from different Grids, from the end user perspective; and (ii) the simultaneous contribution of resources to different Grids, from the resource owner perspective. In this paper, we demonstrate that a decentralized and “end-to-end” scheduling and execution system can efficiently interoperate different Grids. In particular, we evaluate the coordinated use of the EGEE and IRISGrid testbeds in the execution of a Bioinformatics application. Results show the feasibility of building loosely coupled computational Grid environments only based on Globus services, while obtaining non-trivial levels of quality of service, in terms of performance and reliability. Such approach allows a straightforward resource sharing since the resources are accessed by using de facto standard protocols and interfaces.     

Grid Interoperability Based on a Formal Design

We can distinguish two different Grid concepts: desktop and service Grids. Both Grid concepts have their advantages and disadvantages, however these are different. For example desktop Grids are a cost-effective platform, but sometimes unreliable. On the other hand service Grids are highly reliable, but need remarkable funding. The aim of Grid interoperability is to combine the advantages of the different Grid concepts, so the integrated infrastructure offers the best of both concepts. Within the paper we define the Grid interoperability problem, and approximate to the generic architecture through a formal model. We prove formally that the resulting architecture solves the Grid interoperability problem, and is generic enough to interconnect different Grid infrastructures with minor work. We also show in the paper that the formal concept can be applied for creating a gLite to BOINC bridge, and the performance of the core bridge implementation is satisfactory.     

GridBench: A tool for the interactive performance exploration of Grid infrastructures

As Grids rapidly expand in size and complexity, the task of benchmarking and testing, interactive or unattended, quickly becomes unmanageable. In this article we describe the difficulties of testing/benchmarking resources in large Grid infrastructures and we present the software architecture implementation of GridBench, an extensible tool for testing, benchmarking and ranking of Grid resources. We give an overview of GridBench services and tools, which support the easy definition, invocation and management of tests and benchmarking experiments. We also show how the tool can be used in the analysis of benchmarking results and how the measurements can be used to complement the information provided by Grid Information Services and used as a basis for resource selection and user-driven resource ranking. In order to illustrate the usage of the tool, we describe scenarios for using the GridBench framework to perform test/benchmark experiments and analyze the results.     

Achieving Interoperation of Grid Data Resources via Workflow Level Integration

Production Grids are becoming widely utilized by the e-Science community to run computation and data intensive experiments more efficiently. Unfortunately, different production Grid infrastructures are based on different middleware technologies, both for computation and for data access. Although there is significant effort from the Grid community to standardize the underlying middleware, solutions that allow existing non-standard tools to interoperate are one of the major concerns of Grid users today. This paper describes the generic requirements towards the interoperation of Grid data resources within computational workflows, and suggests integration techniques that allow workflow engines to access various heterogeneous data resources during workflow execution. Reference implementations of these techniques are presented and recommendations on their applicability and suitability are made.     

Grid-enabled Virtual Screening Against Malaria

WISDOM is an international initiative to enable a virtual screening pipeline on a Grid infrastructure. Its first attempt was to deploy large scale in silico docking on a public Grid infrastructure. Protein–ligand docking is about computing the binding energy of a protein target to a library of potential drugs using a scoring algorithm. Previous deployments were either limited to one cluster, to Grids of clusters in the tightly protected environment of a pharmaceutical laboratory or to desktop Grids. The first large scale docking experiment ran on the EGEE Grid production service from 11 July 2005 to 19 August 2005 against targets relevant to research on malaria and saw over 41 million compounds docked for the equivalent of 80 years of CPU time. Up to 1,700 computers were simultaneously used in 15 countries around the world. Issues related to the deployment and the monitoring of the in silico docking experiment as well as experience with Grid operation and services are reported in the paper. The main problem encountered for such a large scale deployment was the Grid infrastructure stability. Although the overall success rate was above 80%, a lot of monitoring and supervision was still required at the application level to resubmit the jobs that failed. But the experiment demonstrated how Grid infrastructures have a tremendous capacity to mobilize very large CPU resources for well targeted goals during a significant period of time. This success leads to a second computing challenge targeting avian flu neuraminidase N1.     

A Market-Oriented Grid Directory Service for Publication and Discovery of Grid Service Providers and their Services

The emergence of Grids as a platform for sharing and aggregation of distributed resources increases the need for mechanisms that allow an efficient management of resources. The Grid economy has been identified as one of the potential solutions as it helps in managing the supply and demand for resources and enables sustained sharing of resources by providing economic incentive for Grid resource providers. An economy based Grid computing environment needs to support an infrastructure that enables the creation of a marketplace for meeting of providers and consumers. This paper presents the Grid Market Directory (GMD) that serves as a registry for publication and discovery of Grid service providers and their services.     

On-demand service hosting on production grid infrastructures

The Software as a Service (SaaS) methodology is a key paradigm of Cloud computing. In this paper, we focus on an interesting topic—to dynamically host services on existing production Grid infrastructures. In general, production Grids normally employ a Job-Submission-Execution (JSE) model with rigid access interfaces. In this paper, we implement the Cyberaide onServe, a lightweight middleware with a virtual appliance. The Cyberaide onServe implements the SaaS model on production Grids by translating the SaaS model to the JSE model. The Cyberaide onServe can be deployed on demand in a virtual appliance, host users’ software as a Web service, accept Web service invocations; finally, the Cyberaide onServe can execute them on production Grids. We have deployed the Cyberaide onServe on the TeraGrid and the test results show that the Cyberaide onServe can provide SaaS functionalities with a good performance.     

Assessing Green Strategies in Peer-to-Peer Opportunistic Grids

Opportunistic peer-to-peer (P2P) Grids are distributed computing infrastructures that harvest the idle computing cycles of computing resources geographically distributed. In these Grids, the demand for resources is typically bursty. During bursts of resource demand, many Grid resources are required, but on other occasions they may remain idle for long periods of time. If the resources are kept powered on even when they are neither processing their owners’ workload nor Grid jobs, their exploitation is not efficient in terms of energy consumption. One way to reduce the energy consumed in these idleness periods is to place the computers that form the Grid in a “sleeping” state which consumes less energy. In Grid computing, this strategy introduces a tradeoff between the benefit of energy saving and the associated costs in terms of increasing the job response time, also known as makespan, and reducing the hard disks’ lifetime. To mitigate these costs, it is usually introduced a timeout policy together with the sleeping state, which tries to avoid useless state transitions. In this work, we use simulations to analyze the potential of using sleeping states to save energy in each site of a P2P Grid. Our results show that sleeping states can save energy with low associated impact on jobs’ makespan and hard disks’ lifetime. Furthermore, the best sleeping strategy to be used depends on the characteristics of each individual site, thus, each site should be configured to use the sleeping strategy that best fits its characteristics. Finally, differently from other kinds of Grid infrastructures, P2P Grids can place a machine in sleeping mode as soon as it becomes idle, i.e. it is not necessary to use an aggressive timeout policy. This allows increases on the Grid’s energy saving without impacting significantly the jobs’ makespan and the disks’ lifetime.     

计算网格中资源调度研发现状及展望

网格在解决分散式资源共享方面具有广阔的应用前景,使得它成为下一代互联网瞩目的焦点.而计算网格为资源的需求和供给提供定量分析,激励资源拥有者加入网格的同时保证资源消费者合理有效地使用资源.介绍计算网格通用模型GRACE的基础上,针对该模型中的热点问题--资源管理和调度展开讨论,分析比较当前的几种调度方案,为进一步提出高效完备的资源调度算法作准备.     

Performance analysis of allocation policies for interGrid resource provisioning

Several Grids have been established and used for varying science applications during the last years. Most of these Grids, however, work in isolation and with different utilisation levels. Previous work has introduced an architecture and a mechanism to enable resource sharing amongst Grids. It has demonstrated that there can be benefits for a Grid to offload requests or provide spare resources to another Grid. In this work, we address the problem of resource provisioning to Grid applications in multiple-Grid environments. The provisioning is carried out based on availability information obtained from queueing-based resource management systems deployed at the provider sites which are the participants of the Grids. We evaluate the performance of different allocation policies. In contrast to existing work on load sharing across Grids, the policies described here take into account the local load of resource providers, imprecise availability information and the compensation of providers for the resources offered to the Grid. In addition, we evaluate these policies along with a mechanism that allows resource sharing amongst Grids. Experimental results obtained through simulation show that the mechanism and policies are effective in redirecting requests thus improving the applications’ average weighted response time.     

A Launch-time Scheduling Heuristics for Parallel Applications on Wide Area Grids

Large and dynamic computational Grids, generally known as wide-area Grids, are characterized by a large availability, heterogene- ity on computational resources, and high vari- ability on their status during the time. Such Grid infrastructures require appropriate schedule mechanisms in order to satisfy the application performance requirements (QoS). In this paper we propose a launch-time heuristics to schedule component-based parallel applications on such kind of Grid. The goal of the proposed heuristics is threefold: to meet the minimal task computation- al requirement, to maximize the throughput between communicating tasks, and to evaluate on-the-fly the resource availability to minimize the aging effect on the resources state. We evaluate the proposed heuristics by simulations applying it to a suite of task graphs and Grid platforms randomly generated. Moreover, a further test was conducted to schedule a real application on a real Grid. Experimental results shown that the proposed solution can be a viable one.     

Using a Simple Prioritisation Mechanism to Effectively Interoperate Service and Opportunistic Grids in the EELA-2 e-Infrastructure

Grids currently in production can be broadly classified as either service Grids, composed of dedicated resources, or opportunistic Grids that harvest the computing power of non-dedicated resources when they are idle. While a service Grid provides high and well defined levels of quality of service, an opportunistic Grid provides only a best-effort service. Nevertheless, since opportunistic Grids do not require resources to be fully dedicated to the Grid, they have the potential to assemble a much larger number of resources. Moreover, these Grids cater very well to the execution of the so-called embarrassingly parallel applications, a type of application that is frequently found in practice, and that comprises the largest portion of the typical workload processed in production Grid systems. The EELA-2 e-infrastructure is comprised of a service Grid and an opportunistic Grid that federates computing resources from scientific institutions in both Europe and Latin America. Due to the complementary characteristics of these two types of Grids, a lot of attention has recently been placed in how to interoperate them. In this paper we focus on the less studied problem of assessing the feasibility of such interoperation. We analyse different prioritisation policies that define when the resources of one Grid can be used to run jobs originating from the other. Our results show that in the absence of a suitable prioritisation policy, the benefits that the users of one Grid may have, frequently come with an important negative impact on the users of the other Grid. We also show that a simple reciprocation mechanism is capable of arbitrating the interoperation in such a way that, whenever possible, users profit from the interoperation and, in no case, this benefit leads to a noticeable reduction on the quality of service that the users would experience were the Grids not to interoperate. We conclude discussing how we have implemented, in the context of the EELA-2 project, this prioritisation mechanism, allowing the effective interoperation of a service Grid based on the gLite middleware with an opportunistic Grid that uses the OurGrid middleware.     

A Secure Grid Medical Data Manager Interfaced to the gLite Middleware

The medical community is producing and manipulating a tremendous volume of digital data for which computerized archiving, processing and analysis is needed. Grid infrastructures are promising for dealing with challenges arising in computerized medicine but the manipulation of medical data on such infrastructures faces both the problem of interconnecting medical information systems to Grid middlewares and of preserving patients’ privacy in a wide and distributed multi-user system. These constraints are often limiting the use of Grids for manipulating sensitive medical data. This paper describes our design of a medical data management system taking advantage of the advanced gLite data management services, developed in the context of the EGEE project, to fulfill the stringent needs of the medical community. It ensures medical data protection through strict data access control, anonymization and encryption. The multi-level access control provides the flexibility needed for implementing complex medical use-cases. Data anonymization prevents the exposure of most sensitive data to unauthorized users, and data encryption guarantees data protection even when it is stored at remote sites. Moreover, the developed prototype provides a Grid storage resource manager (SRM) interface to standard medical DICOM servers thereby enabling transparent access to medical data without interfering with medical practice.     

A super-peer model for resource discovery services in large-scale Grids

As deployed Grids increase from tens to thousands of nodes, peer-to-peer (P2P) techniques and protocols can be used to implement scalable services and applications. The super-peer model is a novel approach that helps the convergence of P2P models and Grid environments and can be used to deploy a P2P information service in Grids. A super-peer serves a single physical organization in a Grid, and manages metadata associated to the resources provided by the nodes of that organization. Super-peers connect to each other to form a peer network at a higher level. This paper examines how the super-peer model can handle membership management and resource discovery services in a multi-organizational Grid. A simulation analysis evaluates the performance of a resource discovery protocol; simulation results can be used to tune protocol parameters in order to increase search efficiency.     

Flexible Grid service management through resource partitioning

In this paper, a distributed and scalable Grid service management architecture is presented. The proposed architecture is capable of monitoring task submission behaviour and deriving Grid service class characteristics, for use in performing automated computational, storage and network resource-to-service partitioning. This partitioning of Grid resources amongst service classes (each service class is assigned exclusive usage of a distinct subset of the available Grid resources), along with the dynamic deployment of Grid management components dedicated and tuned to the requirements of a particular service class introduces the concept of Virtual Private Grids. We present two distinct algorithmic approaches for the resource partitioning problem, the first based on Divisible Load Theory (DLT) and the second built on Genetic Algorithms (GA). The advantages and drawbacks of each approach are discussed and their performance is evaluated on a sample Grid topology using NSGrid, an ns-2 based Grid simulator. Results show that the use of this Service Management Architecture in combination with the proposed algorithms improves computational and network resource efficiency, simplifies schedule making decisions, reduces the overall complexity of managing the Grid system, and at the same time improves Grid QoS support (with regard to job response times) by automatically assigning Grid resources to the different service classes prior to scheduling.     

Fault tolerant and prioritized scheduling in OGSA‐based mobile Grids

Grids and mobile Grids can form the basis and the enabling technology for pervasive and utility computing due to their ability to being open, highly heterogeneous and scalable. In this paper we present a scheme for advancing quality of service (QoS) attributes, such as fault tolerance and prioritized scheduling, in OGSA‐based mobile Grids. The fault tolerance is achieved by producing and managing sufficient replicas of tasks submitted for execution on the mobile Grid resources. We design a simple and efficient prioritization scheme, which allows the scheduling of the tasks submitted by the Grid users as distinguished priorities that can be managed and exploited as a QoS parameter by the Grid infrastructure operator. The results that are presented show the efficiency of the proposed scheme in being simple and additionally enriching with reliability and QoS features the applications that are built on the concept of mobile Grids. Copyright © 2008 John Wiley & Sons, Ltd.