Fault-tolerant grid architecture and practice

Grid computing emerges as effective technologies to couple geographically dis-tributed resources and solve large-scale computational problems in wide area networks. The fault tolerance is a significant and complex issue in grid computing systems. Various techniques have been investigated to detect and correct faults in distributed computing systems. Unreliable fault detection is one of the most effective techniques. Globus as a grid middleware manages resources in a wide area network. The Globns fault detection service uses the well-known techniques basedon unreliable fault detectors to detect and report component failures. However, more powerful techniques are required to detect and correct both system-level and application-level faults in agrid system, and a convenient toolkit is also needed to maintain the consistency in the grid. Afault-tolerant grid platform (FTGP) based on an unreliable fault detector and the Globus faultdetection service is presented in this paper. The platform offers effective strategies in such threeaspects as grid key components, user tasks, and high-level applications.     

基于Globus的网格应用关键技术研究

随着网格研究的不断深入和迅速发展,网格的应用领域不断扩大,出现了适用于不同应用领域的网格技术,更广泛地直接服务于生产和各种商业活动.通过介绍网格概念,分析Globus工具包,研究和探讨了在OGSA体系结构和Globus支撑平台下实现网格商业应用的一些关键性技术,如高性能宽带网络、网格体系结构设计、网格操作系统设计、网格安全认证技术以及网格使用模式设计等.     

Design and implementation of a workflow-based resource broker with information system on computational grids

The grid is a promising infrastructure that can allow scientists and engineers to access resources among geographically distributed environments. Grid computing is a new technology which focuses on aggregating resources (e.g., processor cycles, disk storage, and contents) from a large-scale computing platform. Making grid computing a reality requires a resource broker to manage and monitor available resources. This paper presents a workflow-based resource broker whose main functions are matching available resources with user requests and considering network information statuses during matchmaking in computational grids. The resource broker provides a graphic user interface for accessing available and the appropriate resources via user credentials. This broker uses the Ganglia and NWS tools to monitor resource status and network-related information, respectively. Then we propose a history-based execution time estimation model to predict the execution time of parallel applications, according to previous execution results. The experimental results show that our model can accurately predict the execution time of embarrassingly parallel applications. We also report on using the Globus Toolkit to construct a grid platform called the TIGER project that integrates resources distributed across five universities in Taichung city, Taiwan, where the resource broker was developed.

The application hosting environment: Lightweight middleware for grid-based computational science

Grid computing is distributed computing performed transparently across multiple administrative domains. Grid middleware, which is meant to enable access to grid resources, is currently widely seen as being too heavyweight and, in consequence, unwieldy for general scientific use. Its heavyweight nature, especially on the client-side, has severely restricted the uptake of grid technology by computational scientists. In this paper, we describe the Application Hosting Environment (AHE) which we have developed to address some of these problems. The AHE is a lightweight, easily deployable environment designed to allow the scientist to quickly and easily run legacy applications on distributed grid resources. It provides a higher level abstraction of a grid than is offered by existing grid middleware schemes such as the Globus Toolkit. As a result, the computational scientist does not need to know the details of any particular underlying grid middleware and is isolated from any changes to it on the distributed resources. The functionality provided by the AHE is ‘application-centric’: applications are exposed as web services with a well-defined standards-compliant interface. This allows the computational scientist to start and manage application instances on a grid in a transparent manner, thus greatly simplifying the user experience. We describe how a range of computational science codes have been hosted within the AHE and how the design of the AHE allows us to implement complex workflows for deployment on grid infrastructure.     

A computational economy for grid computing and its implementation in the Nimrod-G resource broker

Computational grids that couple geographically distributed resources such as PCs, workstations, clusters, and scientific instruments, have emerged as a next generation computing platform for solving large-scale problems in science, engineering, and commerce. However, application development, resource management, and scheduling in these environments continue to be a complex undertaking. In this article, we discuss our efforts in developing a resource management system for scheduling computations on resources distributed across the world with varying quality of service (QoS). Our service-oriented grid computing system called Nimrod-G manages all operations associated with remote execution including resource discovery, trading, scheduling based on economic principles and a user-defined QoS requirement. The Nimrod-G resource broker is implemented by leveraging existing technologies such as Globus, and provides new services that are essential for constructing industrial-strength grids. We present the results of experiments using the Nimrod-G resource broker for scheduling parametric computations on the World Wide Grid (WWG) resources that span five continents.     

网格技术的发展及其应用研究

网格计算是一个新出现的名词，也是近年来兴起的一个研究热点，它是以大规模的资源协作共享、创新的应用以及高性能计算为特点，诞生的一个全新领域。本文在阅读大量最新的国内外相关文献后，对网格计算的概念、特点、基本体系结构进行简要介绍，并详细分析了网格的研究现状、应用领域以及发展趋势。     

一种新的网格环境模型——TGrid Model

在分析了现有网格环境不足的基础上，提出一种新的网格环境模型——基于树形结构的网格体系与环境TGrid，支持高性能计算，面向主题的资源共享和新一代的需求建模。它以树结构来组织网格节点和集成各种资源，实现了自底向上、多级、面向需求的资源抽象和多种资源融合。而且树型结构符合自然层次组织关系，容易实现网格系统的层次化管理，有利于减轻中心节点的负载和实现大规模应用的负载平衡，提高资源查找效率。同时，TGrid以虚拟资源的形式实现网格资源的共享，利用分布式JVM（TJVM）虚拟网格节点上CPU和主存资源，利用多数据库中间件（TDOD）实现数据库级资源集成和共享，利用Globus网格服务（GService）实现其他软件和数据资源共享。该树型网格为日益增长的网格应用的需求提供了新的解决方案。     

网格计算的研究新进展

网格计算诞生了一个全新的领域,它以大规模的资源协作共享、创新的应用以及高性能计算的特点,区别于传统的分布式计算。文章简述了网格计算的定义、特点、功能和基本体系结构,重点综述了网格的研究新进展及其商业化应用前景,分析了当前网格研究的发展趋势及其我国的对策。     

DynaGrid: A dynamic service deployment and resource migration framework for WSRF-compliant applications

Large-scale Grid is a computing environment composed of Internet-wide distributed resources shared by a number of applications. Although WSRF and Java-based hosting environment can successfully deal with the heterogeneity of resources and the diversity of applications, the current Grid systems have several limitations to support the dynamic nature of large-scale Grid.This paper proposes DynaGrid, a new framework for building large-scale Grid for WSRF-compliant applications. Compared to the existing Grid systems, DynaGrid provides three new mechanisms: dynamic service deployment, resource migration, and transparent request dispatching. Two core components, ServiceDoor and dynamic service launcher (DSL), have been implemented as WSRF-compliant Web services to realize DynaGrid, which are applicable to any Java-based WSRF hosting environment. We construct a real testbed with DynaGrid on the Globus Toolkit 4 and evaluate the effectiveness of our framework using two practical applications. The evaluation results show that dynamic service deployment and resource migration in DynaGrid bring many advantages to large-scale Grid in terms of performance and reliability with minimal overhead.     

REMUS: A Rerouting and Multiplexing System for Grid Connectivity Across Firewalls

The Grid provides unique opportunities for high-performance computing through distributed applications that execute over multiple remote resources. Participating institutions can form a virtual organization to maximize the utilization of collective resources as well as to facilitate collaborative projects. However, there are two design aspects in distributed environments like the Grid that can easily clash: security and resource sharing. It may be that resources are secure but are not entirely conducive to resource sharing, or networks are wide open for resource sharing but sacrifice security as a result. We developed REMUS, a rerouting and multiplexing system that provides a compromise through connection rerouting and wrappers. REMUS reroutes connections using proxies, ports and protocols that are already authorized across firewalls, avoiding the need to make new openings through the firewalls. We also encapsulate applications within wrappers, transparently rerouting the connections among Grid applications without modifying their programs. In this paper, we describe REMUS and the tests we conducted across firewalls using two Grid middleware case studies: Globus Toolkit 2.4 and Nimrod/G 3.0.     

Adaptive computing on the Grid using AppLeS

Ensembles of distributed, heterogeneous resources, also known as computational grids, have emerged as critical platforms for high-performance and resource-intensive applications. Such platforms provide the potential for applications to aggregate enormous bandwidth, computational power, memory, secondary storage, and other resources during a single execution. However, achieving this performance potential in dynamic, heterogeneous environments is challenging. Recent experience with distributed applications indicates that adaptivity is fundamental to achieving application performance in dynamic grid environments. The AppLeS (Application Level Scheduling) project provides a methodology, application software, and software environments for adaptively scheduling and deploying applications in heterogeneous, multiuser grid environments. We discuss the AppLeS project and outline our findings.     

地震勘探集成环境资源监控研究与实现

利用网格技术建设地震勘探集成环境高性能计算平台基础设施的过程中,监控系统不仅要为位于不同管理域、分布异构的高性能资源管理提供资源全局视图,方便对资源管理和控制,还要为分布环境下资源的选择、任务的调度与执行提供指导信息．本文提出了面向地震勘探领域的高性能应用网格监控系统体系结构,建立了监控信息数据模型,在GlobusToolkit信息服务组件的基础上扩展了信息采集功能和信息查询接口,并基于Websphere为高性能计算资源的信息查询与监控提供了资源实体的统一视图和可视化管理Portal．经测试,系统运行稳定,可扩展性强．     

面向调度主站一体化的数据网格设计与实现

高性能电力应用致力于搭建用于调度数据的统一管理、集中处理平台。数据网格易于海量数据共享的特点使其适用于实现调度主站一体化。构建了调度主站数据网格的基本架构,利用网格软件Globus Toolkit建立网格环境,在此基础上配置数据库访问中间件OGSA-DAI来建立面向调度主站一体化的数据网格实验平台。     

基于Globus Toolkit4的网格多资源Web服务实现

采用Globus作为事实上的网格基础平台,基于Globus进行网格服务开发是当前开发网格应用系统的主要手段.它提供了网格系统的基础服务,可以构建复杂的网格应用系统.针对GlobusToolkit4中网格服务的实现问题,介绍了单资源服务、多资源服务的实现模型,并以一个简单的网格计算为例,详细地说明了网格多资源Web服务的实现过程,最后成功地在单资源网格服务的基础上实现了网格多资源服务.     

Supporting dynamic access to virtualized data resources via WSRF-based services

Today grid applications require not only lots of computational power but data at a very large scale too. Although grid computing was initially conceptualized as the technology that focuses on solving compute-intensive problems, this focus has gradually shifted to applications where data is distributed over various locations. Access to these data resources stored in heterogeneous grid storage systems located at geographically distributed virtual organizations in an integrated and uniform way is a challenging problem. The Web Services Resource Framework (WSRF) has recently emerged as the standard for the development and integration of grid services. This paper proposes and presents Gravy4WS, a middleware architecture based on WSRF Web services that enables the dynamic access to virtualized grid data resources. A novel scheduling algorithm called DCE (Delegating-Cluster-Execution based Scheduling) is proposed to improve load balancing of the system. The implementation of Gravy4WS using WSRF libraries and services provided by Globus Toolkit 4 is described together with its performance evaluation.     

基于Agent的网格互连结构

目前有许多论坛、试验环境和研究项目都在进行网格技术的研究，但这些研究都自成体系，采用的技术各不相同，这些网格系统不能互连、互通、互操作．为了解决这一问题，本文提出基于Agent的网格互连结构，把Agent技术和网格技术结合起采，对网格互连的安全机制和资源管理机制进行了研究，并给出了网格互连结构的设计．在本文提出的结构解决了Inter—Domain网格安全机制不同的问题、Inter—Domain网格资源共享的问题，可实现Inter—Domain网格单点登录（Single—Sign—On）和代表（Delegation），具有通用、简单、高效、分布式等优点．     

基于Globus的网格安全机制研究及扩展

随着网格应用的发展,网格安全问题日渐突出。为解决此问题,Globus项目组作了大量工作。讨论网格环境下的安全问题,分析Globus的安全机制,结合虚拟组织技术和基于角色的访问控制方法,探讨基于Globus安全基础设施的扩展的安全技术——社区授权服务机制,并对网格环境下的安全技术的基本问题进行了总结。     

网格技术及其应用

网格实现了将计算机网络作为统一的计算资源的可能性。这是一种动态的、多机构虚拟组织的资源协调共享和问题解决的崭新技术，与传统分布式计算的区别，主要在于其重点是大规模的资源共享、创新的应用以及高性能的目标。文章阐述了网格的基本概念，并介绍了网格的应用与发展。     

网格环境下的一种事务协调机制及其补偿技术

事务处理是网格计算能否投入实际应用尤其是商业应用的关键技术之一，用于网格环境下的事务处理必须同时提供协调短时操作和长时间的商业活动的能力，基于Gtobus Toolkit和代理技术，提出了一种能够同时管理原子事务和聚合事务的事务协调机制，研究了相应的可以容错各种失败的协调算法，通过自动产生和执行补偿事务，聚合事务协调机制可以满足网格计算中对长事务的处理要求。     

网格体系结构及关键技术分析

网格计算是以大规模的资源协作共享、创新的应用以度高性能计算为特点，诞生的一个全新领域。本文对网格的特点和体系结构进行了介绍，并详细分析了网格计算的关键技术。