A standards‐based Grid resource brokering service supporting advance reservations,coallocation, and cross‐Grid interoperability

The problem of Grid‐middleware interoperability is addressed by the design and analysis of a feature‐rich, standards‐based framework for all‐to‐all cross‐middleware job submission. The architecture is designed with focus on generality and flexibility and builds on extensive use, internally and externally, of (proposed) Web and Grid services standards such as WSRF, JSDL, GLUE, and WS‐Agreement. The external use provides the foundation for easy integration into specific middlewares, which is performed by the design of a small set of plugins for each middleware. Currently, plugins are provided for integration into Globus Toolkit 4 and NorduGrid/ARC. The internal use of standard formats facilitates customization of the job submission service by replacement of custom components for performing specific well‐defined tasks. Most importantly, this enables the easy replacement of resource selection algorithms by algorithms that address the specific needs of a particular Grid environment and job submission scenario. By default, the service implements a decentralized brokering policy, striving to optimize the performance for the individual user by minimizing the response time for each job submitted. The algorithms in our implementation perform resource selection based on performance predictions, and provide support for advance reservations as well as coallocation of multiple resources for coordinated use. The performance of the system is analyzed with focus on overall service throughput (up to over 250 jobs per min) and individual job submission response time (down to under 1 s). Copyright © 2009 John Wiley & Sons, Ltd.     

Ontology‐based Grid resource management

Grid resources are typically diverse in nature with respect to their software and hardware configurations, resource usage policies and the kind of application they support. Aggregating and monitoring these resources, and discovering suitable resources for the applications become a challenging issue. This is partially due to the representation of Grid metadata supported by the existing Grid middleware which offers limited scope for matching the job requirements that directly affect scheduling decisions. This paper proposes a semantic component in conventional Grid architecture to support ontology‐based representation of Grid metadata and facilitate context‐based information retrieval that complements Grid schedulers for effective resource management. Web Ontology language is used for creating Grid resource ontology and Algernon inference engine has been used for resource discovery. This semantic component has been integrated with conventional Grid schedulers. Several experiments have also been carried out to investigate the performance overhead that arises while integrating this component with Grid schedulers. Copyright © 2009 John Wiley & Sons, Ltd.     

Flexible Grid service management through resource partitioning

In this paper, a distributed and scalable Grid service management architecture is presented. The proposed architecture is capable of monitoring task submission behaviour and deriving Grid service class characteristics, for use in performing automated computational, storage and network resource-to-service partitioning. This partitioning of Grid resources amongst service classes (each service class is assigned exclusive usage of a distinct subset of the available Grid resources), along with the dynamic deployment of Grid management components dedicated and tuned to the requirements of a particular service class introduces the concept of Virtual Private Grids. We present two distinct algorithmic approaches for the resource partitioning problem, the first based on Divisible Load Theory (DLT) and the second built on Genetic Algorithms (GA). The advantages and drawbacks of each approach are discussed and their performance is evaluated on a sample Grid topology using NSGrid, an ns-2 based Grid simulator. Results show that the use of this Service Management Architecture in combination with the proposed algorithms improves computational and network resource efficiency, simplifies schedule making decisions, reduces the overall complexity of managing the Grid system, and at the same time improves Grid QoS support (with regard to job response times) by automatically assigning Grid resources to the different service classes prior to scheduling.     

Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions

We present algorithms, methods, and software for a Grid resource manager, that performs resource brokering and job scheduling in production Grids. This decentralized broker selects computational resources based on actual job requirements, job characteristics, and information provided by the resources, with the aim to minimize the total time to delivery for the individual application. The total time to delivery includes the time for program execution, batch queue waiting, and transfer of executable and input/output data to and from the resource. The main features of the resource broker include two alternative approaches to advance reservations, resource selection algorithms based on computer benchmark results and network performance predictions, and a basic adaptation facility. The broker is implemented as a built-in component of a job submission client for the NorduGrid/ARC middleware.     

网格数据访问中间件的设计与实现

网格服务的一个重要目标是实现数据资源的共享,然而网格环境的复杂性阻碍了对数据的有效访问。利用中间件屏蔽底层的复杂性,实现对网格环境下跨平台,异构数据资源的统一访问是当前研究的重点。该文结合中科院科学数据网格项目的特点,阐述了在开放网格服务架构下用中间件来实现网格中数据的统一访问的原理,并给出了实现的方法。     

面向高性能计算环境的作业优化调度模型的设计与实现

高性能计算环境聚合了多个分布在不同地域、不同组织机构的高性能计算资源,面向用户提供统一的访问入口和使用方式,由系统中间件根据用户作业请求匹配合适的高性能计算资源。随着环境应用编程接口的开放以及作业请求数量的大幅增加,面对高并发作业提交请求时,目前采用的即时调度模型会由于网络等原因导致一定数量的请求处理失败,同时缺乏灵活性。针对此问题,优化了环境作业调度模型,引入作业环境队列,细化了作业系统层状态,增加了作业调度策略可配置性,并基于环境中间件SCE实现了系统原型。经测试,在单核心服务每分钟处理近200个作业提交请求的工作负载下,无因系统和网络原因引起的作业提交出错现象;在共计1 000个作业中,近500个作业提交命令请求在0.3s以内完成,800余个作业提交命令请求在0.5s以内完成。     

Achieving efficiency, quality of service and robustness in multi-organizational Grids

Scalability, flexibility, quality of service provisioning, efficiency and robustness are the desired characteristics of most computing systems. Although the emerging Grid computing paradigm is scalable and flexible, achieving both efficiency and quality of service provisioning in Grids is a challenging task but is necessary for the wide adoption of Grids. Grid middleware should also be robust to uncertainties such as those in user-estimated runtimes of Grid applications. In this paper, we present a complete middleware framework for Grids that achieves user satisfaction by providing QoS guarantees for Grid applications, cost effectiveness by efficiently utilizing resources and robustness by intelligently handling uncertain runtimes of applications.     

A framework for adaptive execution in grids

Grids offer a dramatic increase in the number of available processing and storing resources that can be delivered to applications. However, efficient job submission and management continue being far from accessible to ordinary scientists and engineers due to their dynamic and complex nature. This paper describes a new Globus based framework that allows an easier and more efficient execution of jobs in a ‘submit and forget’ fashion. The framework automatically performs the steps involved in job submission and also watches over its efficient execution. In order to obtain a reasonable degree of performance, job execution is adapted to dynamic resource conditions and application demands. Adaptation is achieved by supporting automatic application migration following performance degradation, ‘better’ resource discovery, requirement change, owner decision or remote resource failure. The framework is currently functional on any Grid testbed based on Globus because it does not require new system software to be installed in the resources. The paper also includes practical experiences of the behavior of our framework on the TRGP and UCM‐CAB testbeds. Copyright © 2004 John Wiley & Sons, Ltd.     

一个应用于中文信息处理的网格系统的设计

介绍了目前中文信息处理研究的不足和网格技术的发展.为了扩大计算和存储的共享规模,并充分复用各类技术和研究,提出了将中文信息处理的应用利用网格技术来实现的方案.描述了一个应用于中文信息处理的网格系统的整体框架,该框架分为四层:顶部是一个Web门户,中间是中文信息处理网格管理器和网格中间件,最下面是网格节点.另外,还分析了负责资源管理和作业分配的CIPG管理器的设计方案.     

Introduce: An Open Source Toolkit for Rapid Development of Strongly Typed Grid Services

Service-oriented architectures and applications have gained wide acceptance in the Grid computing community. A number of tools and middleware systems have been developed to support application development using Grid Services architectures. Most of these efforts, however, have focused on low-level support for management and execution of Grid services, management of Grid-enabled resources, and deployment and execution of applications that make use of Grid services. Simple-to-use service development tools, which would allow a Grid service developer to leverage Grid technologies without needing to know low-level details, are becoming increasingly important for wider application of the Grid. In this paper, we describe an open-source, extensible toolkit, called Introduce, that supports easy development and deployment of Web Services Resource Framework (WSRF) compliant services. Introduce is designed to reduce the service development and deployment effort by hiding low level details of the Globus Toolkit and to enable the implementation of strongly typed services. In strongly typed services, a service produces and consumes data types that are well-defined and published in the Grid. This enables data-level syntactic interoperability so that clients and services can access and consume data elements programmatically and correctly. We expect that enabling strongly typed Grid services while lowering the difficulty of entry to the Grid via toolkits like Introduce will have a major impact to the success of the Grid and its wider adoption as a viable technology of choice in the commercial sector as well as in academic, medical, and government research.     

Service level agreement based adaptive Grid superscheduling

Grid computing brings heterogeneity and decentralization to the world of science and technology. It leverages every bit of idle computing resources and provides a straightforward middleware for integrating cross-domain scientific devices and legacy systems. In a super big Grid, job scheduling is challenging specifically when it needs to have access to vast amount of resources. The process of mapping jobs onto Grid resources requires significant consideration in terms of Grid architecture design, consumer demands and provider revenues. In this paper, we simultaneously utilize the legacy architecture of superscheduling, forwarding strategy, service level, success rate, and service pricing strategies and finally propose a service level agreement based on adaptive superscheduling (SAS) algorithm. SAS algorithm presents unified connectivity via efficient diffusion of jobs through the Grid infrastructure that is fueled from the previous scheduling events across the Grid. Moreover, by enforcing the service level agreement terms from a rich set of ask and bid prices, system performance, and load statistics, SAS successfully boosts revenue and utilization statistics. We perform an extensive experimental analysis for different Grid scales. Based on our experimental result, the SAS algorithm maximizes revenue while guarantees quality of service. More specifically, the quality of service is achieved through a high ratio of completed jobs and remarkable utilization of resources.     

数据网格信息服务体系的研究与设计

网格信息服务是网格中间件的核心部分之一,提供各种分布式网格资源的静态与动态信息。现有的网格中间件几乎都是针对计算网格设计的,它们所提供的信息服务不能胜任数据网格的要求。论文主要对数据网格中的信息服务体系进行研究和设计。     

基于GOS的国家网格集成环境及应用实例开发*

中国国家网格作为国内最主要的网格为用户提供了良好的计算服务.详细讨论了网格中间件系统软件GOS的功能和结构,以及基于GOS的中国国家网格集成环境,给出了生物信息软件MEME在国家网格环境下的集成实现,并提出了今后的工作方向.     

Triana Applications within Grid Computing and Peer to Peer Environments

An overview of the Triana Problem Solving Environment is provided – with a particular focus on the GAP application-level interface, for integration with Grid Computing and Peer-to-Peer infrastructure. GAP is a Java-based subset of the Grid Application Toolkit interface (being implemented in the GridLab project), and an outline of its current functionality, usage and mappings to three supported underlying middleware derivatives: JXTA, Web Services, and P2PS (a simplified Peer-to-Peer platform) are provided. The motivation behind the development of P2PS is given – emphasising its minimal, but effective Peer-to-Peer mechanisms that allow scalable, decentralized discovery and communication amongst cooperating P2PS peers within highly unstable environments. A summary of three application use cases illustrating the range of scenarios that such a system addresses is also provided.     

Pro-active failure handling mechanisms for scheduling in grid computing environments

In this paper, we consider designing pro-active failure handling strategies for grid environments. These strategies estimate the availability of resources in the Grid, and also preemptively calculate the expected long term capacity of the Grid. Using these strategies, we create modified versions of the backfill and replication algorithms to include all three pro-active strategies to ascertain each of their effectiveness in the prevention of job failures during execution. Also, we extend our earlier work on a co-ordinate based allocation strategy. The extended algorithm also shows continual improvement when operating under the same execution environment. In our experiments, we compare these enhanced algorithms to their original forms, and show that pro-active failure handling is able to, in some cases, avoid all job failures during execution. Also, we show that NSA provides the best balance of enhanced throughput and job failures during execution of the algorithms we have considered.     

一个高分子模拟计算网格的作业管理

高分子模拟计算网格为复杂材料研究人员提供了分子模拟软件、计算资源和信息共享的平台。设计和实现了网格作业管理模块,包括作业的提交、调度和监视,为用户提供了一个透明的网格资源的使用方法。该模块已经成功部署到高分子模拟计算网格系统中,试验结果表明,作业管理为网格用户提供更好的服务质量,实现了对网格资源的优化使用。     

An Information Service for Grid Virtual Organization: Architecture, Implementation and Evaluation

Information Service is the heart of the entire Grid software infrastructure. It maintains various information in a Grid environment, and provides this information to users on request. We present an Information Service architecture for information capturing, aggregation, and provisioning in a Grid Virtual Organization (VO). This Information Service is a hierarchical structure which consists of VO layer, site layer and resource layer. The architecture defines downloadable and pluggable information sensors for portability and flexibility of information capturing. The meta-data mechanism is incorporated into our Information Service for information organization and management. Experiments were conducted to evaluate the performance of the Information Service, and the results show that the Information Service presents satisfactory scalability with number of users and number of information amount. Our Information Service has been implemented based on the Globus Toolkit as a Grid service compliant to the Open Grid Services Architecture (OGSA) specifications.     

中国国家网格作业管理设计与实现

基于中国国家网格（CNGrid）及其系统软件CNGrid GOS,提出并实现了一种VegaBatch网格作业管理系统。VegaBatch通过基于队列长度预测的元调度策略解决了网格作业调度问题;利用双重同步机制维护全局网格作业信息视图;借助用户身份切换机制保障作业数据安全;增加作业重执行功能提供数据后处理能力。对比了国际上较为成熟的网格作业管理软件,讨论了VegaBatch解决的问题。该软件目前已部署在CNGrid环境,提供了功能丰富、安全可靠的网格作业管理功能,降低了CNGrid的使用管理成本,提升了用户体验。     

基于.net的校园网格异构数据统一访问接口

针对开放网格服务架构-数据访问与接口中间件只面向JDBC的局限性,提出构建一个基于.net架构的异构数据统一访问接口中间件。采用XML Web Service技术跨平台性和ADO.net增强的数据库访问能力,既实现了接口实现的通用性与易扩展性,又实现了接口对数据访问、数据库连接、数据格式转换、数据传输、数据库集成等底层细节的屏蔽。测试结果表明,该接口中间件具有通用性、灵活性和可靠性。     

Enabling Interoperability among Grid Meta-Schedulers

The goal of Grid computing is to integrate the usage of computer resources from cooperating partners in the form of Virtual Organizations (VO). One of its key functions is to match jobs to execution resources efficiently. For interoperability between VOs, this matching operation occurs in resource brokering middleware, commonly referred to as the meta-scheduler or meta-broker. In this paper, we present an approach to a meta-scheduler architecture, combining hierarchical and peer-to-peer models for flexibility and extensibility. Interoperability is further promoted through the introduction of a set of protocols, allowing meta-schedulers to maintain sessions and exchange job and resource state using Web Services. Our architecture also incorporates a resource model that enables an efficient resource matching across multiple Virtual Organizations, especially where the compute resources and state are dynamic. Experiments demonstrate these new functional features across three distributed organizations (BSC, FIU, and IBM), that internally use different job scheduling technologies, computing infrastructure and security mechanisms. Performance evaluations through actual system measurements and simulations provide the insights on the architecture’s effectiveness and scalability.