Efficient extension of gLite VOs with BOINC based desktop grids

The paper describes the results of the EU FP7 EDGI project concerning how to extend gLite VOs with public and institutional BOINC desktop grids. Beyond simply showing the integration architecture components and services, the main emphasis is on how this integrated architecture can efficiently support parameter study applications, based on the so-called metajob concept created by the EDGI project. The paper explains in detail how to use the metajob concept by gLite users to exploit the BOINC desktop grids connected to the gLite VO, as well as how metajobs are managed internally by the 3G Bridge service. Performance measurements show that the Metajob concept indeed can significantly improve the performance of gLite VOs extended with desktop grids. Finally, the paper describes the practical ways of connecting BOINC desktop grids to gLite VOs and the accounting mechanism in these integrated grid systems.     

EDGeS: Bridging EGEE to BOINC and XtremWeb

Desktop Grids, such as XtremWeb and BOINC, and Service Grids, such as EGEE, are two different approaches for science communities to gather computing power from a large number of computing resources. Nevertheless, little work has been done to combine these two Grid technologies in order to establish a seamless and vast Grid resource pool. In this paper we present the EGEE Service Grid, the BOINC and XtremWeb Desktop Grids. Then, we present the EDGeS solution to bridge the EGEE Service Grid with the BOINC and XtremWeb Desktop Grids.     

Boosting gLite with cloud augmented volunteer computing

The paper details the result of the EU FP7 EDGI project focusing on the cloud developments and usability improvements. Volunteer desktop grids, like BOINC, are designed to handle millions of parameter sweep type jobs and millions of desktop machines as worker nodes. Seamless transfer of gLite jobs to desktop grids was already implemented by EDGI; however this huge number of DG resources could not be utilized efficiently due to slow completion time caused by unpredictable behavior of the volunteer resources. The paper details how clouds have been utilized to shorten completion time on the EDGeS@home volunteer desktop grid to boost the performance of the supported gLite VO and also details how this service can be exploited by the gLite user communities of EGI (European Grid Initiative) all over Europe.     

Grid Interoperability Based on a Formal Design

We can distinguish two different Grid concepts: desktop and service Grids. Both Grid concepts have their advantages and disadvantages, however these are different. For example desktop Grids are a cost-effective platform, but sometimes unreliable. On the other hand service Grids are highly reliable, but need remarkable funding. The aim of Grid interoperability is to combine the advantages of the different Grid concepts, so the integrated infrastructure offers the best of both concepts. Within the paper we define the Grid interoperability problem, and approximate to the generic architecture through a formal model. We prove formally that the resulting architecture solves the Grid interoperability problem, and is generic enough to interconnect different Grid infrastructures with minor work. We also show in the paper that the formal concept can be applied for creating a gLite to BOINC bridge, and the performance of the core bridge implementation is satisfactory.     

From Dedicated Grid to Volunteer Grid: Large Scale Execution of a Bioinformatics Application

Large volunteer desktop platforms are now available for several kind of applications. More and more scientists consider this type of computing power as an alternative to the classical platforms such as dedicated clusters aggregated into Grids. This paper presents the work we did to run the first phase of the Help Cure Muscular Dystrophy project to run on World Community Grid. The project was launched on December 19, 2006, and took 26 weeks to complete. During this time frame, 123 GB of results were produced by volunteers who share their idle CPU time to compute a cross docking experiment over 168 proteins. We present performance evaluation of the overall execution and compare the World Community Grid volunteer Grid with a dedicated one.     

Using a Simple Prioritisation Mechanism to Effectively Interoperate Service and Opportunistic Grids in the EELA-2 e-Infrastructure

Grids currently in production can be broadly classified as either service Grids, composed of dedicated resources, or opportunistic Grids that harvest the computing power of non-dedicated resources when they are idle. While a service Grid provides high and well defined levels of quality of service, an opportunistic Grid provides only a best-effort service. Nevertheless, since opportunistic Grids do not require resources to be fully dedicated to the Grid, they have the potential to assemble a much larger number of resources. Moreover, these Grids cater very well to the execution of the so-called embarrassingly parallel applications, a type of application that is frequently found in practice, and that comprises the largest portion of the typical workload processed in production Grid systems. The EELA-2 e-infrastructure is comprised of a service Grid and an opportunistic Grid that federates computing resources from scientific institutions in both Europe and Latin America. Due to the complementary characteristics of these two types of Grids, a lot of attention has recently been placed in how to interoperate them. In this paper we focus on the less studied problem of assessing the feasibility of such interoperation. We analyse different prioritisation policies that define when the resources of one Grid can be used to run jobs originating from the other. Our results show that in the absence of a suitable prioritisation policy, the benefits that the users of one Grid may have, frequently come with an important negative impact on the users of the other Grid. We also show that a simple reciprocation mechanism is capable of arbitrating the interoperation in such a way that, whenever possible, users profit from the interoperation and, in no case, this benefit leads to a noticeable reduction on the quality of service that the users would experience were the Grids not to interoperate. We conclude discussing how we have implemented, in the context of the EELA-2 project, this prioritisation mechanism, allowing the effective interoperation of a service Grid based on the gLite middleware with an opportunistic Grid that uses the OurGrid middleware.     

Role-based access control for a Grid system using OGSA-DAI and Shibboleth

In this paper, we propose a new role-based access control (RBAC) system for Grid data resources in the Open Grid Services Architecture Data Access and Integration (OGSA-DAI). OGSA-DAI is a widely used framework for integrating data resources in Grids. However, OGSA-DAI’s identity-based access control causes substantial administration overhead for the resource providers in virtual organizations (VOs) because of the direct mapping between individual Grid users and the privileges on the resources. To solve this problem, we used the Shibboleth, an attribute authorization service, to support RBAC within the OGSA-DAI. In addition, access control policies need to be specified and managed across multiple VOs. For the specification of access control policies, we used the Core and Hierarchical RBAC profile of the eXtensible Access Control Markup Language (XACML); and for distributed administration of those policies and the user-role assignments, we used the Object, Metadata and Artifacts Registry (OMAR). OMAR is based on the e-business eXtensible Markup Language (ebXML) registry specifications developed to achieve interoperable registries and repositories. Our RBAC system provides scalable and fine-grain access control and allows privacy protection. It also supports dynamic delegation of rights and user-role assignments, and reduces the administration overheads for the resource providers because they need to maintain only the mapping information from VO roles to local database roles. Moreover, unnecessary mapping and connections can be avoided by denying invalid requests at the VO level. Performance analysis shows that our RBAC system adds only a small overhead to the existing security infrastructure of OGSA-DAI.     

The Design, Usage, and Performance of GRUBER: A Grid Usage Service Level Agreement based BrokERing Infrastructure

We present GRUBER, a Grid Resource Usage service level agreement (uSLA) based BrokERing infrastructure, aimed at addressing the challenging issues that can arise within virtual organizations (VOs) that integrate participants and resources spanning multiple physical administrative domains. In such environments, participants delegate to one or more VOs the right to use certain resources subject to local policy and service level agreements; each VO then uses those resources subject to VO policy. GRUBER supports the explicit representation, enforcement, and management of service level agreements (SLAs) concerning resource usage (uSLAs) that can serve as an objective organizing principle for controlled resource sharing in distributed systems. uSLAs express how resources must be used over various time intervals and represent a novelty for the Grid domain. This paper provides a detailed overview of the GRUBER infrastructure, the evolution of its design to improve scalability, specifically the distribution of the resource brokering service, and the extended support for dynamic environments. We also present various results achieved over time that demonstrate both the utility and performance of GRUBER under various application workloads and scenarios. This work was carried out for CoreGrid IST project n°004265, funded by the European Commission.     

Grid-enabled Virtual Screening Against Malaria

WISDOM is an international initiative to enable a virtual screening pipeline on a Grid infrastructure. Its first attempt was to deploy large scale in silico docking on a public Grid infrastructure. Protein–ligand docking is about computing the binding energy of a protein target to a library of potential drugs using a scoring algorithm. Previous deployments were either limited to one cluster, to Grids of clusters in the tightly protected environment of a pharmaceutical laboratory or to desktop Grids. The first large scale docking experiment ran on the EGEE Grid production service from 11 July 2005 to 19 August 2005 against targets relevant to research on malaria and saw over 41 million compounds docked for the equivalent of 80 years of CPU time. Up to 1,700 computers were simultaneously used in 15 countries around the world. Issues related to the deployment and the monitoring of the in silico docking experiment as well as experience with Grid operation and services are reported in the paper. The main problem encountered for such a large scale deployment was the Grid infrastructure stability. Although the overall success rate was above 80%, a lot of monitoring and supervision was still required at the application level to resubmit the jobs that failed. But the experiment demonstrated how Grid infrastructures have a tremendous capacity to mobilize very large CPU resources for well targeted goals during a significant period of time. This success leads to a second computing challenge targeting avian flu neuraminidase N1.     

Multi-Grid, Multi-User Workflows in the P-GRADE Grid Portal

Computational Grids connect resources and users in a complex way in order to deliver nontrivial qualities of services. According to the current trend various communities build their own Grids and due to the lack of generally accepted standards these Grids are usually not interoperable. As a result, large scale sharing of resources is prevented by the isolation of Grid systems. Similarly, people are isolated, because the collaborative work of Grid users is not supported by current environments. Each user accesses Grids as an individual person without having the possibility of organizing teams that could overcome the difficulties of application development and execution more easily. The paper describes a new workflow-oriented portal concept that solves both problems. It enables the interoperability of various Grids during the execution of workflow applications, and supports users to develop and run their Grid workflows in a collaborative way. The paper also introduces a classification model that can be used to identify workflow-oriented Grid portals based on two general features: Ability to access multiple Grids, and support for collaborative problem solving. Using the approach the different potential portal types are introduced, their unique features are discussed and the portals and Problem Solving Environments (PSE) of our days are classified. The P-GRADE Portal as a Globus-based implementation for the classification model is also presented. The work described in this paper is supported by the Hungarian Grid project (IHM 4671/1/2003), by the Hungarian OTKA project (No. T042459) and a collaboration project with the University of Reading.     

Fault tolerant and prioritized scheduling in OGSA‐based mobile Grids

Grids and mobile Grids can form the basis and the enabling technology for pervasive and utility computing due to their ability to being open, highly heterogeneous and scalable. In this paper we present a scheme for advancing quality of service (QoS) attributes, such as fault tolerance and prioritized scheduling, in OGSA‐based mobile Grids. The fault tolerance is achieved by producing and managing sufficient replicas of tasks submitted for execution on the mobile Grid resources. We design a simple and efficient prioritization scheme, which allows the scheduling of the tasks submitted by the Grid users as distinguished priorities that can be managed and exploited as a QoS parameter by the Grid infrastructure operator. The results that are presented show the efficiency of the proposed scheme in being simple and additionally enriching with reliability and QoS features the applications that are built on the concept of mobile Grids. Copyright © 2008 John Wiley & Sons, Ltd.     

PaGrid: A Mesh Partitioner for Computational Grids

Computational Grids are emerging as a new infrastructure for high performance computing. Since the resources in a Grid can be heterogeneous and distributed, mesh-based applications require a mesh partitioner that considers both processor and network heterogeneity. We have developed a heterogeneous mesh partitioner, called PaGrid. PaGrid uses a multilevel graph partitioning approach, augmented by execution time load balancing in the final uncoarsening phase. We show that minimization of total communication cost (e.g., as used by JOSTLE) can lead to significant load being placed on processors connected by slow links, which results in higher application execution times. Therefore, PaGrid balances the estimated execution time of the application across processors. PaGrid performance is compared with two existing mesh partitioners, METIS 4.0 and JOSTLE 3.0, for mapping several application meshes to two models of heterogeneous computational Grids. PaGrid is found to produce significantly better partitions than JOSTLE and slightly better partitions than METIS in most cases, in terms of estimated application execution time averaged over a large number of runs with different random number seeds.     

Scheduling Task Parallel Applications for Rapid Turnaround on Enterprise Desktop Grids

Desktop Grids are popular platforms for high throughput applications, but due to their inherent resource volatility it is difficult to exploit them for applications that require rapid turnaround. Efficient desktop Grid execution of short-lived applications is an attractive proposition and we claim that it is achievable via intelligent resource selection. We propose three general techniques for resource selection: resource prioritization, resource exclusion, and task duplication. We use these techniques to instantiate several scheduling heuristics. We evaluate these heuristics through trace-driven simulations of four representative desktop Grid configurations. We find that ranking desktop resources according to their clock rates, without taking into account their availability history, is surprisingly effective in practice. Our main result is that a heuristic that uses the appropriate combination of resource prioritization, resource exclusion, and task replication can achieve performance within a factor of 1.7 of optimal in practice.     

Dynamic scheduling for heterogeneous Desktop Grids

Desktop Grids have emerged as an important methodology to harness the idle cycles of millions of participant desktop PCs over the Internet. However, to effectively utilize the resources of a Desktop Grid, it is necessary to use scheduling policies suitable for such systems. In this paper, we analyze the performance of a policy which is shown to perform well in highly heterogeneous Desktop Grids. The policy utilizes the solution to a linear programming (LP) problem which maximizes system capacity. We suggest robust modifications to address several limitations of the policy.     

Role-Based Access Control in a Data Grid Using the Storage Resource Broker and Shibboleth

In this paper, we propose a role-based access control (RBAC) system for data resources in the Storage Resource Broker (SRB). The SRB is a Data Grid management system, which can integrate heterogeneous data resources of virtual organizations (VOs). The SRB stores the access control information of individual users in the Metadata Catalog (MCAT) database. However, because of the specific MCAT schema structure, this information can only be used by the SRB applications. If VOs also have many non-SRB applications, each with its own storage format for user access control information, it creates a scalability problem with regard to administration. To solve this problem, we developed a RBAC system with Shibboleth, which is an attribute authorization service currently being used in many Grid environments. Thus, the administration overhead is reduced because the role privileges of individual users are now managed by Shibboleth, not by MCAT or applications. In addition, access control policies need to be specified and managed across multiple VOs. For the specification of access control policies, we used the Core and Hierarchical RBAC profile of the eXtensible Access Control Markup Language (XACML); and for distributed administration of those policies, we used the Object, Metadata and Artifacts Registry (OMAR). OMAR is based on the e-business eXtensible Markup Language (ebXML) registry specifications developed to achieve interoperable registries and repositories. Our RBAC system provides scalable and fine-grain access control and allows privacy protection. Performance analysis shows that our system adds only a small overhead to the existing security infrastructure of the SRB.     

A scalable blackbox-oriented e-learning system based on desktop grid over private cloud

Traditional web-based e-learning system suffers from unstable workloads and security risks of incorporating external executable objects to servers. This paper addresses these issues with emerging technologies, as desktop grid and cloud computing. Learning users are motivated to be volunteers by hosting the virtual machines equipped with e-learning desktop grid applications. We develop components to integrate the e-learning system and desktop grid into the circumstance in which each user serves not only a task producer, but also a volunteer that solves tasks. In order to enhance the responsiveness between the passive desktop grid server and e-learning system, we have also developed asynchronous processes to enable the server and volunteer workers to cooperate in a tightly coupled manner. The system achieves the scalability by maintaining the ratio between the number of volunteers and the number of online users beyond certain threshold.     

Decentralized proactive resource allocation for maximizing throughput of P2P Grid

Peer-to-peer Desktop Grids provide integrated computational resources by leveraging autonomous desktop computers located at the edge of the Internet to offer high computing power. The arbitrary arrival and serving rates of tasks on peers impedes the high throughput in large-scale P2P Grids. We propose a novel autonomous resource allocation scheme, which can maximize the throughput of self-organizing P2P Grid systems. Our design possesses three key features: (1) high adaptability to dynamic environment by proactive and convex-optimal estimation of nodes’ volatile states; (2) minimized task migration conflict probability (upper bound can be limited to 2%) of over-utilized nodes individually shifting surplus loads; (3) a load-status conscious gossip protocol for optimizing distributed resource discovery effect. Based on a real-life user’s workload and capacity distribution, the simulation results show that our approach could get significantly improved throughput with 23.6–47.1% reduction on unprocessed workload compared to other methods. We also observe high scalability of our solution under dynamic peer-churning situations.     

Cross‐domain authorization for federated virtual organizations using the myVocs collaboration environment

This paper describes our experiences building and working with the reference implementation of myVocs (my Virtual Organization Collaboration System). myVocs provides a flexible environment for exploring new approaches to security, application development, and access control built from Internet services without a central identity repository. The myVocs framework enables virtual organization (VO) self‐management across unrelated security domains for multiple, unrelated VOs. By leveraging the emerging distributed identity management infrastructure. myVocs provides an accessible, secure collaborative environment using standards for federated identity management and open‐source software developed through the National Science Foundation Middleware Initiative. The Shibboleth software, an early implementation of the Organization for the Advancement of Structured Information Standards Security Assertion Markup Language standard for browser single sign‐on, provides the middleware needed to assert identity and attributes across domains so that access control decisions can be determined at each resource based on local policy. The eduPerson object class for lightweight directory access protocol (LDAP) provides standardized naming, format, and semantics for a global identifier. We have found that a Shibboleth deployment supporting VOs requires the addition of a new VO service component allowing VOs to manage their own membership and control access to their distributed resources. The myVocs system can be integrated with Grid authentication and authorization using GridShib. Copyright © 2008 John Wiley & Sons, Ltd.     

基于重复数据删除的虚拟桌面存储优化技术

虚拟桌面基础架构依靠数据中心海量的云基础设施,为用户按需提供虚拟桌面部署所需的软硬件资源,但同时面临存储资源利用率低和虚拟机启动慢的困境.针对虚拟桌面存储中具有大量数据冗余的特性,采用重复数据删除技术缩减虚拟桌面基础架构的存储空间需求;并利用服务器本地磁盘缓存以及共享存储池内的固态硬盘来优化虚拟机的启动性能.通过原型实现,发现相比于基于内容分块的策略,静态分块策略更适合虚拟桌面存储进行重复数据删除,最优的分块大小为4KB,并能够缩减85%的存储空间容量;通过服务器本地磁盘缓存和基于闪存的固态硬盘进行I/O优化,虚拟机的启动速度能够获得35%的提升.     

Modelling of a self-led critical friend topology in inter-cooperative grid communities

For decades, much work has been done to increase the effectiveness and efficiency of job sharing amongst available computational resources. Resources can be organized into a variety of topologies, and recent work has shown that a decentralized distributed resource topology is a crucial but complicated scenario. This is because decentralized resources are normally grouped into independent virtual organizations (VOs) and isolated from each other by VO boundaries.To convey jobs across gaps between various virtual organizations, a novel resource topology called the self-led critical friend model (CFM) is proposed in this work. The CFM deals with trust credits between resources according to their historical collaboration records. This trust reveals a feasible, realistic, and transferable correlation to facilitate the resource selection process for job delegation between arbitrarily connected physical resources. Consequently, the CFM is able to overcome the constraints caused by virtual organization boundaries.Besides the theoretical model, a simulation-based implementation is carried out together with a complementary high-level community-aware scheduling protocol. After evaluating a number of compositional scenarios and criteria objectives, the observed results show the benefit of job scheduling across multiple VOs, as well as the capability of the self-led critical friend model as a novel cross-VO resource topology to represent and interconnect resources.