Workflow-Based Data Parallel Applications on the EGEE Production Grid Infrastructure

Setting up and deploying complex applications on a Grid infrastructure is still challenging and the programming models are rapidly evolving. Efficiently exploiting Grid parallelism is often not straight forward. In this paper, we report on the techniques used for deploying applications on the EGEE production Grid through four experiments coming from completely different scientific areas: nuclear fusion, astrophysics and medical imaging. These applications have in common the need for manipulating huge amounts of data and all are computationally intensive. All the cases studied show that the deployment of data intensive applications require the development of more or less elaborated application-level workload management systems on top of the gLite middleware to efficiently exploit the EGEE Grid resources. In particular, the adoption of high level workflow management systems eases the integration of large scale applications while exploiting Grid parallelism transparently. Different approaches for scientific workflow management are discussed. The MOTEUR workflow manager strategy to efficiently deal with complex data flows is more particularly detailed. Without requiring specific application development, it leads to very significant speed-ups.     

Searching for Software on the EGEE Infrastructure

Several large-scale Grid infrastructures are currently in operation around the world, federating an impressive collection of computational resources, a wide variety of application software, and hundreds of user communities. To better serve the current and prospective users of Grid infrastructures, it is important to develop advanced software retrieval services that could help users locate software components suitable to their needs. In this paper, we present the design and implementation of Minersoft, a distributed, multi-threaded harvester for application software located in large-scale Grid infrastructures. Minersoft crawls the sites of a Grid infrastructure, discovers installed software resources, annotates them with keyword-rich metadata, and creates inverted indexes that can be used to support full-text software retrieval. We present insights derived from the implementation and deployment of Minersoft on EGEE, one of the largest Grid production services currently in operation. Experimental results show that Minersoft achieves a high performance in crawling EGEE sites and discovering software-related files, and a high efficiency in supporting software retrieval.     

Practical Experience from Porting and Executing the Wien2k Application on the EGEE Production Grid Infrastructure

While the Grid promises to deliver a large number of computation nodes to a user, this computation power is not usable without the proper adaption of the application for the Grid. In this paper, we describe the methods used to port and execute a particular application, Wien2k, on the EGEE production Grid. First, the process of porting the application is described. Then, we investigate the measures necessary to execute the application in this production Grid environment efficiently. Although the focus is on this special application, we describe generic methods which can be applied to all applications. We specifically address: Creating a workflow from an application and mapping this workflow to a Grid workflow using the activity attraction pattern. We discuss workflow engines which support cycles in their application workflow. We investigate naïve and worker scheduling techniques. A short introduction into licensing on the Grid is given. Optimisation techniques such as deployment re-use are discussed. Different data transfer mechanisms, centralised data transfer, data re-use, storage element data transfer, and peer-to-peer data transfer are compared. The paper is concluded with suggestions for further workflow porting.     

Infrastructure Federation Through Virtualized Delegation of Resources and Services

Infrastructure federation is becoming an increasingly important issue for modern Distributed Computing Infrastructures (DCIs): Dynamic elasticity of quasi-static Grid environments, incorporation of special-purpose resources into commoditized Cloud infrastructures, cross-community collaboration for increasingly diverging areas of modern e-Science, and Cloud Bursting pose major challenges on the technical level for many resource and middleware providers. Especially with respect to increasing costs of operating data centers, the intelligent yet automated and secure sharing of resources is a key factor for success. With the D-Grid Scheduler Interoperability (DGSI) project within the German D-Grid Initiative, we provide a strategic technology for the automatically negotiated, SLA-secured, dynamically provisioned federation of resources and services for Grid-and Cloud-type infrastructures. This goal is achieved by complementing current DCI schedulers with the ability to federate infrastructure for the temporary leasing of resources and rechanneling of workloads. In this work, we describe the overall architecture and SLA-secured negotiation protocols within DGSI and depict an advanced mechanism for resource delegation through means of dynamically provisioned, virtualized middleware. Through this methodology, we provide the technological foundation for intelligent capacity planning and workload management in a cross-infrastructure fashion.     

Optimization of Jobs Submission on the EGEE Production Grid: Modeling Faults Using Workload

It is commonly observed that production Grids are inherently unreliable. The aim of this work is to improve Grid application performances by tuning the job submission system. A stochastic model, capturing the behavior of a complex Grid workload management system is proposed. To instantiate the model, detailed statistics are extracted from dense Grid activity traces. The model is exploited for optimizing a simple job resubmission strategy. It provides quantitative inputs to improve job submission performance and it enables the impact of faults and outliers on Grid operations to be quantified.     

Coordinated harnessing of the IRISGrid and EGEE testbeds with GridWay

Since the late 1990s, we have witnessed an extraordinary development of Grid technologies. Nowadays, different Grids are being deployed within the context of a growing number of national and transnational research projects. However, the coexistence of those different infrastructures involves two challenging issues, namely: (i) simultaneous and coordinated use of resources from different Grids, from the end user perspective; and (ii) the simultaneous contribution of resources to different Grids, from the resource owner perspective. In this paper, we demonstrate that a decentralized and “end-to-end” scheduling and execution system can efficiently interoperate different Grids. In particular, we evaluate the coordinated use of the EGEE and IRISGrid testbeds in the execution of a Bioinformatics application. Results show the feasibility of building loosely coupled computational Grid environments only based on Globus services, while obtaining non-trivial levels of quality of service, in terms of performance and reliability. Such approach allows a straightforward resource sharing since the resources are accessed by using de facto standard protocols and interfaces.     

Web Services构架下的空间应用集成框架

基于XML,Web Service,Ontology等技术的PAI集成框架包括Web应用层、集成层、分布的服务层和分布的信息层4层．其中集成层含有集成总线及Adapter Service、元数据库和集成协调器与供二次开发的API及Web Service工具集,它实现了数据互操作、软件互操作与语义互操作．集成总线是一组面向领域的编码标准与通信协议,是集成框架的核心;与总线标准不符的遗留系统和其他相关系统只需编写．Adapter Service就能被集成;元数据库用于管理系统中的所有信息与应用服务,屏蔽信息及服务的分布性与异构性;集成协调器用于数据的更新和元数据的管理;Ontology用于定义描述领域相关元数据的术语;数据交换格式与通信协议均用XML定义;所有应用通过SOAP协议实现互操作．该集成框架实现了大规模空间信息及服务的动态集成,能保护已有投资,并能集成网上丰富的商业Web Service,是一个开放的可扩充的框架．文中还介绍了PAI集成框架实现的初步成果,并给出了该框架的一个应用实例——基于PAI集成框架的基于位置的服务。     

A Scalable Cluster-based Infrastructure for Edge-computing Services

In this paper we present a scalable and dynamic intermediary infrastructure, SEcS (acronym of “Scalable Edge computing Services”), for developing and deploying advanced Edge computing services, by using a cluster of heterogeneous machines. Our goal is to address the challenges of the next-generation Internet services: scalability, high availability, fault-tolerance and robustness, as well as programmability and quick prototyping. The system is written in Java and is based on IBM's Web Based Intermediaries (WBI) [71] developed at IBM Almaden Research Center.     

云计算的基础结构设计和云应用服务

云计算是基于云结构的信息技术资源消费与利用系统,也是基于互联网的链路传输系统与终端服务系统的集成应用模型。典型的云服务模型包括Saa S/Aaa S、Iaa S、Paa S、Daa S,这些参考模型同时组成云系统应用服务基础架构。云计算开发标准与规则是云结构发展与云数据库的开发参考基准,也是云管理与云进化的参考结构。云系统服务质量关系云应用发展,而云管理的服务配置与监视服务提供反馈信息。云应用服务维护运行与数据安全是云安全提供的基础保障服务。云备份作为领导性企业云的高级服务功能应能适应云灵活多变性与容灾免灾设计,文中深入分析云数据备份技术应用与程序算法,细述实用数据应用处理方法。动态开发环境是企业云对云计算开放研究平台,在全球推进云环境的开发测试。云计算风险分析与管理同时促进云应用进化。     

EDGeS: Bridging EGEE to BOINC and XtremWeb

Desktop Grids, such as XtremWeb and BOINC, and Service Grids, such as EGEE, are two different approaches for science communities to gather computing power from a large number of computing resources. Nevertheless, little work has been done to combine these two Grid technologies in order to establish a seamless and vast Grid resource pool. In this paper we present the EGEE Service Grid, the BOINC and XtremWeb Desktop Grids. Then, we present the EDGeS solution to bridge the EGEE Service Grid with the BOINC and XtremWeb Desktop Grids.     

基于网格的NGN增值业务生成框架

下一代网络（NGN）最大的特点之一是具备强大的增值业务提供能力，如何快速有效地生成、部署新的增值业务是当前研究的热点。该文以开放业务规范Parlay／OSAAPI为基础，探讨了基于网格技术的NGN增值业务生成框架，以实现多网络融合的灵活业务生成，并以一个第3方呼叫业务为例进行了说明。     

移动互联网中间层基础服务平台的架构和设计

为了更好地服务移动互联网中手机终端软件,提高手机终端软件的品质和业务需要,移动互联网中间层基础服务平台应运而生,它是手机终端调用各个合作商平台的基础,是实现版本升级和后台维护的保证,是完成复杂业务逻辑的引擎,是数据融合,云能力,LBS等创新产品的源头.     

Dynamic Partitioning of GATE Monte-Carlo Simulations on EGEE

The EGEE Grid offers the necessary infrastructure and resources for reducing the running time of particle tracking Monte-Carlo applications like GATE. However, efforts are required to achieve reliable and efficient execution and to provide execution frameworks to end-users. This paper presents results obtained with porting the GATE software on the EGEE Grid, our ultimate goal being to provide reliable, user-friendly and fast execution of GATE to radiation therapy researchers. To address these requirements, we propose a new parallelization scheme based on a dynamic partitioning and its implementation in two different frameworks using pilot jobs and workflows. Results show that pilot jobs bring strong improvement w.r.t. regular gLite submission, that the proposed dynamic partitioning algorithm further reduces execution time by a factor of two and that the genericity and user-friendliness offered by the workflow implementation do not introduce significant overhead.     

Consuming Resources and Services from Multiple Clouds

The consumption of resources and services from multiple Clouds for reasons like high availability, cost reductions or special features is a natural evolution from in-silo Clouds. Several middleware are already available for multiple Clouds. However, due to the complexity of the technical solutions, their approaches are quite different and a classification is needed to guide the potential users. This paper looks to the reports on multiple Cloud topics and proposes a specific taxonomy. It identifies the ready-to-use software and services and classifies them according the taxonomy. It also underlines the driving needs and requirements from consumers’ and providers’ point of views. A particular Cloudware is provided as an example for the degree of requirements fulfillment.     

Centralizing Information About Library Services and Resources

SUMMARY

The School of Information Resources and Library Science at the University of Arizona (UA) has a mix of local and distance students in its program. The librarian at the UA Library who works with this program developed a Web site that centralizes information about several library services and resources that benefit distance learners: online orientation and research guides, chat reference, document delivery, and more. Bringing together this information with a focus on distance learners provides a platform for delivering the library to all users, whatever their distance from the library.     

网格计算环境配置及资源创建

网格技术近年来发展迅猛,已被人们视为继Internet、Web之后的第三大技术浪潮。而Globus Toolkit4．0（GT4）现已成为构建网格环境和开发网格服务的首选平台。该论文讨论了网格服务的基本思想,分析了GT4的新特性及其在产业界的应用,探讨了在Windowsxp上安装和配置GT4的方法和过程,并且介绍了使用网格模拟器GridSim创建一个网格资源的过程。     

网格计算环境配置及资源创建

网格技术近年来发展迅猛,已被人们视为继Internet、Web之后的第三大技术浪潮。而Globus Toolkit4.0(GT4)现已成为构建网格环境和开发网格服务的首选平台。该论文讨论了网格服务的基本思想,分析了GT4的新特性及其在产业界的应用,探讨了在Windows xp上安装和配置GT4的方法和过程,并且介绍了使用网格模拟器GridSim创建一个网格资源的过程。     

Web Services实现校园网信息资源共享

提出信息资源共享和统一身份认证的必要性。总结了在数字化校园系统开发中应用WebServices的优势。介绍了与面向服务架构的WebServices技术相关的XML、SOAP、WSDL、UDDI等协议和技术标准。并以学生信息查询为例阐述了在．net平台上开发WebServices的过程。     

基于Web服务的分布式教学资源集成方案研究

在分布异构自治的环境中如何集成分布式教学资源是一个急需解决课题.文中首先讨论了 Web Services概念、应用体系结构,然后提出基于XML的教学资源描述方法和利用J2EE/Web services实现教学资源集成的框架.