基于校园网络的元计算实验系统WADE的设计与实现

元计算系统是可以作为虚拟的整体而使用的地理上分散的异构计算资源，这些资源包括计算机、数据库和昂贵仪器等．元计算系统在硬件和软件方面均有异构性，适合具有不同内在并行性的复杂应用的执行．现存的绝大多数并行系统都是同构的，不具有这一优势，因此，研究异构环境下的元计算系统就很有现实意义．WADE是基于校园网络开发的元计算实验系统，使用MD支持异构数据格式转换，使用面向对象技术实现单一映像系统，使用优先约束的任务调度算法来实现应用程序的调度和运行，并提供与流行的并行编程软件如PVM等的接口。     

Resource management in Legion

The recent development of gigabit networking technology, combined with the proliferation of low-cost, high-performance microprocessors, has given rise to metacomputing environments. These environments can combine many thousands of hosts, from hundreds of administrative domains, connected by transnational and world-wide networks. Managing the resources in such a system is a complex task, but is necessary to efficiently and economically execute user programs.

In this paper, we describe the resource management portions of the Legion metacomputing system, including the basic model and its implementation. These mechanisms are flexible both in their support for system-level resource management but also in their adaptability for user-level scheduling policies. We show this by implementing a simple scheduling policy and demonstrating how it can be adapted to more complex algorithms.     

计算网格中多编程环境的研究与实现

计算网格(也称元计算系统)聚集地理上分散的资源进行大型的分布式高性能计算。PVM和MPI是广泛使用的并行编程环境,它们需要作为并行计算的基本构建而集成到元计算系统中去。论文针对元计算资源的动态性、分布性、性能多变性和结点异构性等特点,实现了一个自适应的、一体化的多编程环境。论文论述了该多编程环境的体系结构,并利用代理技术实现远程编译、发现资源、屏蔽异构和优化调度。     

WAMM in the framework of graphical user interfaces for metacomputing management

Current environments for metacomputing generally have tools for managing the resources of a metacomputer but often lack adequate tools for designing, writing, and executing programs. Building an application for a metacomputer typically involves writing source codes on a local node, transferring and compiling codes on every node, and starting their execution. Without such tools, the application development phases can come up against considerable difficulties. In order to alleviate these problems, some graphical user interfaces (GUIs) based on PVM, such as XPVM, Parallel Application Development Environment (PADE) and Wide Area Metacomputing Manager (WAMM) have been implemented. These GUIs integrate a programming environment which facilitates the user in performing the application development phases and the application execution.

This paper outlines the general requirements for designing GUIs for metacomputing management, and compares WAMM, a graphical user interface, with some related works.     

A new model of security for metasystems

With the rapid growth of high-speed networking and microprocessing power, metasystems have become increasingly popular. The need for protection and security in such environments has never been greater. However, the conventional approach to security, that of enforcing a single system-wide policy, will not work for the large-scale distributed systems we envision. Our new model shifts the emphasis from ‘system as enforcer’ to user-definable policies, making users responsible for the security of their objects. This security model has been implemented as part of the Legion project. Legion is an object-oriented metacomputing system, with strong support for autonomy. This includes support for per-object, user-defined policies in many areas, including resource management and security. This paper briefly describes the Legion system, presents our security model, and discusses the realization of that model in Legion.     

Adaptive computing on the Grid using AppLeS

Ensembles of distributed, heterogeneous resources, also known as computational grids, have emerged as critical platforms for high-performance and resource-intensive applications. Such platforms provide the potential for applications to aggregate enormous bandwidth, computational power, memory, secondary storage, and other resources during a single execution. However, achieving this performance potential in dynamic, heterogeneous environments is challenging. Recent experience with distributed applications indicates that adaptivity is fundamental to achieving application performance in dynamic grid environments. The AppLeS (Application Level Scheduling) project provides a methodology, application software, and software environments for adaptively scheduling and deploying applications in heterogeneous, multiuser grid environments. We discuss the AppLeS project and outline our findings.     

Message-passing environments for metacomputing

In this paper, we present the three libraries PACX-MPI, PLUS, and PVMPI that provide message-passing between different high-performance computers in metacomputing environments. Each library supports the development and execution of distributed metacomputer applications.

The PACX-MPI approach offers a transparent interface for the communication between two or more MPI environments. PVAMPI allows the user spawning parallel processes under the MPI environment. The PLUS protocol bridges the gap between vendor-specific (e.g., MPL, NX, and PARIX) and vendor-independent message-passing environments (e.g., PVM and MPI). Moreover, it offers the ability to create and control processes at application runtime.     

基于XML的分布式虚拟运行环境建模与实现

当前很多网络计算环境尚缺乏为多种应用提供隔离、定制的运行环境的能力,给网络计算环境的广泛应用带来障碍.而虚拟机技术的飞速发展,使得根据用户需求,按需构建隔离、定制的分布式虚拟运行环境成为可能.基于XML schema技术对分布式虚拟运行环境进行了建模和描述,阐述了分布式虚拟运行环境的运行支撑系统(ACOPE系统)的详细设计和原型实现.该系统能够通过对宿主资源和虚拟机资源的有效管理,根据分布式虚拟运行环境的XML描述文件,在网络计算环境中选取部分宿主资源并在其上创建或克隆满足相应配置需求的虚拟机,支持分布式虚拟运行环境的动态按需构建和使用.最后通过实验验证了ACOPE系统的有效性.     

Scalable networked information processing environment (SNIPE)

Scalable Networked Information Processing Environment (SNIPE) is a metacomputing system that aims to provide a reliable, secure, fault-tolerant environment for long-term distributed computing applications and data stores across the global Internet. This system combines global naming and replication of both processing and data to support large-scale information processing applications leading to better availability and reliability than currently available with typical cluster computing and/or distributed computer environments. To facilitate this the system supports: distributed data collection, distributed computation, distributed control and resource management, distributed output and process migration. The underlying system supports multiple communication paths, media and routing methods to aid performance and robustness across both local and global networks. This paper details the goals, design and an initial implementation of SNIPE, and then demonstrates its usefulness in supporting a middleware project. Initial communications performance is also presented.     

A metacomputing environment for demanding applications: design, implementation, experiments and business benefit

The paper describes the design, implementation, and use of a commercial metacomputing environment for computationally intensive loosely-coupled parallel applications. Much weight has been laid on practical and commercialisation aspects, and on business benefit. This distinguishes this work from many other metacomputing activities in a positive way. It demonstrates how a metacomputing environment can be used to improve a company’s position in the market. A cluster of networked geographically dispersed computing nodes is considered as physical layer. The proposed distribution of work over the nodes of the execution network is proven optimal, in terms of minimizing the execution time, with respect to the availability of resources. We also present our experience on testing the environment for computing-intensive 3D-rendering jobs derived from the ESPRIT project EROPPA and demonstrate that the new environment can change dramatically the character of the post production business.     

Dynamic Configuration for Distributed Systems

Dynamic system configuration is the ability to modify and extend a system while it is running. The facility is a requirement in large distributed systems where it may not be possible or economic to stop the entire system to allow modification to part of its hardware or software. It is also useful during production of the system to aid incremental integration of component parts, and during operation to aid system evolution. The paper introduces a model of the configuration process which permits dynamic incremental modification and extension. Using this model we determine the properties required by languages and their execution environments to support dynamic configuration. CONIC, the distributed system which has been developed at Imperial College with the specific objective of supporting dynamic configuration, is described to illustrate the feasibility of the model.     

An elasticity model for High Throughput Computing clusters

Different methods have been proposed to dynamically provide scientific applications with execution environments that hide the complexity of distributed infrastructures. Recently virtualization has emerged as a promising technology to provide such environments. In this work we present a generic cluster architecture that extends the classical benefits of virtual machines to the cluster level, so providing cluster consolidation, cluster partitioning and support for heterogeneous environments. Additionally the capacity of the virtual clusters can be supplemented with resources from a commercial cloud provider. The performance of this architecture has been evaluated in the execution of High Throughput Computing workloads. Results show that, in spite of the overhead induced by the virtualization and cloud layers, these virtual clusters constitute a feasible and performing HTC platform. Additionally, we propose a performance model to characterize these variable capacity (elastic) cluster environments. The model can be used to dynamically dimension the cluster using cloud resources, according to a fixed budget, or to estimate the cost of completing a given workload in a target time.     

Metadata for Managing Grid Resources in Data Mining Applications

The Grid is an infrastructure for resource sharing and coordinated use of those resources in dynamic heterogeneous distributed environments. The effective use of a Grid requires the definition of metadata for managing the heterogeneity of involved resources that include computers, data, network facilities, and software tools provided by different organizations. Metadata management becomes a key issue when complex applications, such as data-intensive simulations and data mining applications, are executed on a Grid. This paper discusses metadata models for heterogeneous resource management in Grid-based data mining applications. In particular, it discusses how resources are represented and managed in the Knowledge Grid, a framework for Grid-enabled distributed data mining. The paper illustrates how XML-based metadata is used to describe data mining tools, data sources, mining models, and execution plans, and how metadata is used for the design and execution of distributed knowledge discovery applications on Grids.     

PARDIS: Programmer-level abstractions for metacomputing

The potential offered by metacomputing is hard to realize due to the complexity of programming geographically distributed applications spanning different software systems. This paper describes PARDIS, a system designed to address this challenge, based on ideas underlying the Common Object Request Broker Architecture (CORBA), a successful industry standard. PARDIS is a distributed environment in which objects representing data-parallel computations, called Single Program Multiple Data (SPMD) objects, as well as non-parallel objects present in parallel programs, can interact with each other across platforms and software systems. Each of these objects represents a small encapsulated application and can be used as a building block in the construction of powerful distributed metaapplications. The objects interact through interfaces specified in the Interface Definition Language (IDL), which allows the programmer to integrate within one metaapplication component implemented using different software systems. Further, support for non-blocking interactions between objects allows PARDIS to build concurrent distributed scenarios.     

DisCComp – A Formal Model for Distributed Concurrent Components

Most large-scaled software systems are structured in distributed components to manage complexity and have to cope with concurrent executed threads. System decomposition and concurrent flow of execution are orthogonal. A sound semantic model that is powerful enough to handle distributed concurrent components but also realistic enough to provide a foundation for component technologies actually in use is still missing. Therefore, the paper introduces such an operational semantics for distributed concurrent component-based systems. Based on this formal model, UML-based modeling techniques are introduced. Tool support for modeling, code generation, and system execution is provided.     

A computational economy for grid computing and its implementation in the Nimrod-G resource broker

Computational grids that couple geographically distributed resources such as PCs, workstations, clusters, and scientific instruments, have emerged as a next generation computing platform for solving large-scale problems in science, engineering, and commerce. However, application development, resource management, and scheduling in these environments continue to be a complex undertaking. In this article, we discuss our efforts in developing a resource management system for scheduling computations on resources distributed across the world with varying quality of service (QoS). Our service-oriented grid computing system called Nimrod-G manages all operations associated with remote execution including resource discovery, trading, scheduling based on economic principles and a user-defined QoS requirement. The Nimrod-G resource broker is implemented by leveraging existing technologies such as Globus, and provides new services that are essential for constructing industrial-strength grids. We present the results of experiments using the Nimrod-G resource broker for scheduling parametric computations on the World Wide Grid (WWG) resources that span five continents.     

Model continuity in the design of dynamic distributed real-time systems

Model continuity refers to the ability to transition as much as possible a model specification through the stages of a development process. In this paper, the authors show how a modeling and simulation environment, based on the discrete event system specification formalism, can support model continuity in the design of dynamic distributed real-time systems. In designing such systems, the authors restrict such continuity to the models that implement the system's real-time control and dynamic reconfiguration. The proposed methodology supports systematic modeling of dynamic systems and adopts simulation-based tests for distributed real-time software. Model continuity is emphasized during the entire process of software development $the control models of a dynamic distributed real-time system can be designed, analyzed, and tested by simulation methods, and then smoothly transitioned from simulation to distributed execution. A dynamic team formation distributed robotic system is presented as an example to show how model continuity methodology effectively manages the complexity of developing and testing the control software for this system.     

JiniSolve:一种基于Jini的元计算架构的基本体系结构及其初步应用

随着不同种类体系结构高性能计算机系统的不断出现和网络技术的迅猛发展,人们越来越希望能够打破平台和地域的限制,共享分布在广域网上的各种高性能计算机的计算能力。但广域网络分布式计算平台的异构性和资源组织协调的复杂性一直困扰着这方面工作的发展。作者从Java字节码的平台无关性和Jini架构的网络资源运行时动态自组织能力中得到启发,提出了JiniSolve网络异构平台分布式计算架构(一种元计算架构),以克服这两个障碍。JiniSolve架构能够支持所有三种网络分布式计算的模式———远程计算、MobileAgent计算和代码下载计算,其中对MobileAgent计算的支持是以往类似元计算架构所不能做到的。同时,JiniSolve架构又具有运行时动态异步自组织能力,从而使整个系统具有更高的可适应性,可用性,可靠性和易用性。文章将介绍JiniSolve架构的基本体系结构,分析其优点并给出一个实验性的应用举例。     

DISCWorld: an environment for service-based matacomputing

We describe our DISCWorld system for wide-area, high-performance metacomputing in which we adopt a high-level, service-based approach. Users’ client programs request combinations of services from a set of server nodes which communicate at a peer-based level. DISCWorld is a constrained metacomputing system, running only the service operations its participating resource administrators have chosen to provide and advertise, and provides a common integration environment for clients to access these services and developers to make them available. We discuss our software architecture and experiences building DISCWorld using Java and CORBA components, and the associated research issues for metacomputing that we are addressing.     

Dynamic balancing of communication and computation load for HLA-based simulations on large-scale distributed systems

Dynamic balancing of computation and communication load is vital for the execution stability and performance of distributed, parallel simulations deployed on the shared, unreliable resources of large-scale environments. High Level Architecture (HLA) based simulations can experience a decrease in performance due to imbalances that are produced initially and/or during run time. These imbalances are generated by the dynamic load changes of distributed simulations or by unknown, non-managed background processes resulting from the non-dedication of shared resources. Due to the dynamic execution characteristics of elements that compose distributed applications, the computational load and interaction dependencies of each simulation entity change during run time. These dynamic changes lead to an irregular load and communication distribution, which increases overhead of resources and latencies. A static partitioning of load is limited to deterministic applications and is incapable of predicting the dynamic changes caused by distributed applications or by external background processes. Therefore, a scheme for balancing the communication and computational load during the execution of distributed simulations is devised in a scalable hierarchical architecture. The proposed balancing system employs local and cluster monitoring mechanisms in order to observe the distributed load changes and identify imbalances, repartitioning policies to determine a distribution of load and minimize imbalances. A migration technique is also employed by this proposed balancing system to perform reliable and low-latency load transfers. Such a system successfully improves the use of shared resources and increases distributed simulations’ performance by minimizing communication latencies and partitioning the load evenly. Experiments and comparative analyses were conducted in order to identify the gains that the proposed balancing scheme provides to large-scale distributed simulations.