Visual Grid Workflow in Triana

In this paper, we describe the graphical abstractions for Grids and services that have been implemented within the Triana problem solving environment. We provide an overview of the ways in which Triana interacts with services (e.g., Web and P2P services) and then how we interact with core Grid components, such as resource managers and data management systems through the extensive use of the GridLab GAT interface. We describe in detail the GAT philosophy and implementation and then show how the various GAT primitives can be represented in an intuitive fashion within a Triana workflow. This approach, which we refer to as the Visual GAT, differs substantially from other approaches because we do not tie our implementation to any specific underlying Grid middleware technologies; rather, we base our implementation on application level requirements and model such primitives from a user’s perspective by hiding as much complexity as possible without undermining the core capabilities required. We provide a use case to demonstrate the Visual GAT implementation and show how legacy applications can seamlessly be distributed and integrated in a dynamic fashion within complex data-driven workflow scenarios.     

Data discovery algorithm for scientific data grid environment

In modern scientific computing communities, scientists are involved in managing massive amounts of very large data collections in a geographically distributed environment. Research in the area of grid computing has given us various ideas and solutions to address these requirements. Data grid mostly deals with large computational problems and provides geographically distributed resources for large-scale data-intensive applications that generate large data sets. Peer-to-peer (P2P) networks have also become a major research topic over the last few years. In a distributed P2P system, a discovery algorithm is required to locate specific information, applications, or users within the system. In this research work, we present our scientific data grid as a large P2P-based distributed system model. By using this model, we study various discovery algorithms for locating data sets in a data grid system. The algorithms we studied are based on the P2P architecture. We investigate these algorithms using our Grid Simulator developed using PARSEC. In this paper, we illustrate our scientific data grid model and our Grid Simulator. We then analyze the performance of the discovery algorithms relative to their average number of hop, success rates and bandwidth consumption.     

电网业务中的海量数据存储技术

随着电网业务中海量数据的增多,其所采用的集中式数据库或大电网统一数据库的数据存储方式已难以满足电网业务与管理的需要. 本文首先着重分析了电网业务中的数据类型与特性,然后在综合考虑国家电网当前旧设备与通信网络结构的基础上,提出了一种适用于电网业务的分布式存储方案,并对其数据访问控制进行了设计;最后,进行了初步验证与对比分析.     

DynaGrid: A dynamic service deployment and resource migration framework for WSRF-compliant applications

Large-scale Grid is a computing environment composed of Internet-wide distributed resources shared by a number of applications. Although WSRF and Java-based hosting environment can successfully deal with the heterogeneity of resources and the diversity of applications, the current Grid systems have several limitations to support the dynamic nature of large-scale Grid.This paper proposes DynaGrid, a new framework for building large-scale Grid for WSRF-compliant applications. Compared to the existing Grid systems, DynaGrid provides three new mechanisms: dynamic service deployment, resource migration, and transparent request dispatching. Two core components, ServiceDoor and dynamic service launcher (DSL), have been implemented as WSRF-compliant Web services to realize DynaGrid, which are applicable to any Java-based WSRF hosting environment. We construct a real testbed with DynaGrid on the Globus Toolkit 4 and evaluate the effectiveness of our framework using two practical applications. The evaluation results show that dynamic service deployment and resource migration in DynaGrid bring many advantages to large-scale Grid in terms of performance and reliability with minimal overhead.     

Grid Data Management: Open Problems and New Issues

Initially developed for the scientific community, Grid computing is now gaining much interest in important areas such as enterprise information systems. This makes data management critical since the techniques must scale up while addressing the autonomy, dynamicity and heterogeneity of the data sources. In this paper, we discuss the main open problems and new issues related to Grid data management. We first recall the main principles behind data management in distributed systems and the basic techniques. Then we make precise the requirements for Grid data management. Finally, we introduce the main techniques needed to address these requirements. This implies revisiting distributed database techniques in major ways, in particular, using P2P techniques. Work partially funded by ARA “Massive Data” of the French ministry of research (project Respire), the European Strep Grid4All project, the CAPES–COFECUB Daad project and the CNPq–INRIA Gridata project.     

网格与P2P计算混合模型研究

网格和P2P(PeertoPeer)都是分布式计算模型,它们的总体目标相似。利用P2P与网格技术之间的协同和互补,构造了一个二层的网格和P2P计算混合模型,上层是网格层,下层是P2P层。P2P层负责消息路由和资源发现,网格层负责为用户提供网格服务。该模型可以解决网格的单点失效、资源搜索等问题,同时增强网格的可扩展性。     

通用知识网格下以用户为中心的数据挖掘本体研究

分布异构的海量数据挖掘是数据挖掘领域急待解决的课题，通用知识网格（UKB）架构模型用于在网格环境下创建大规模的分布式知识发现和知识集成系统。本体服务器是整个架构的核心模块，负责本体的管理和查询。数据挖掘本体服务是本体服务器提供的主要服务。本文主要介绍通用知识网格下以用户为中心的数据挖掘本体的设计和OWL实现。数据挖掘本体可满足各种不同领域、不同层次用户的知识发现服务，使系统具有开放性、可扩展性和高用户可用性。还介绍了一个反洗钱领域数据挖掘解决方案实例。     

Searching for Software on the EGEE Infrastructure

Several large-scale Grid infrastructures are currently in operation around the world, federating an impressive collection of computational resources, a wide variety of application software, and hundreds of user communities. To better serve the current and prospective users of Grid infrastructures, it is important to develop advanced software retrieval services that could help users locate software components suitable to their needs. In this paper, we present the design and implementation of Minersoft, a distributed, multi-threaded harvester for application software located in large-scale Grid infrastructures. Minersoft crawls the sites of a Grid infrastructure, discovers installed software resources, annotates them with keyword-rich metadata, and creates inverted indexes that can be used to support full-text software retrieval. We present insights derived from the implementation and deployment of Minersoft on EGEE, one of the largest Grid production services currently in operation. Experimental results show that Minersoft achieves a high performance in crawling EGEE sites and discovering software-related files, and a high efficiency in supporting software retrieval.     

Distributed data mining on Agent Grid: Issues,platform and development toolkit

Centralized data mining techniques are widely used today for the analysis of large corporate and scientific data stored in databases. However, industry, science, and commerce fields often need to analyze very large datasets maintained over geographically distributed sites by using the computational power of distributed systems. The Grid can play a significant role in providing an effective computational infrastructure support for this kind of data mining. Similarly, the advent of multi-agent systems has brought us a new paradigm for the development of complex distributed applications. During the past decades, there have been several models and systems proposed to apply agent technology building distributed data mining (DDM). Through a combination of these two techniques, we investigated the critical issues to build DDM on Grid infrastructure and design an Agent Grid Intelligent Platform as a testbed. We also implement an integrated toolkit VAStudio for quickly developing agent-based DDM applications and compare its function with other systems.     

Building a Peer-to-peer Information System in Grids via Self-organizing Agents

A Grid information system should rely upon two basic features: the replication and dissemination of information about Grid services and resources, and the distribution of such information among Grid hosts. This paper examines an approach based on ant systems to replicate and map Grid services information on Grid hosts according to the semantic classification of such services. The Ant-based Replication and MApping Protocol (ARMAP) is used to disseminate resource information by a decentralized mechanism, and its effectiveness is evaluated by means of an entropy index. Information is disseminated by agents – ants – that traverse the Grid by exploiting P2P interconnections among Grid hosts. A mechanism inspired by real ants’ pheromone is used by each agent to autonomously drive its behavior on the basis of its interaction with the environment. “Swarm Intelligence” emerges from the activity of a high number of ants. The ARMAP protocol enables the use of a semi-informed search algorithm which can drive query messages towards a cluster of peers having information about resources belonging to the requested class. A simulation analysis has been performed to evaluate the performance of ARMAP.     

Intercontinental Grids: An Infrastructure for Demand-Driven Innovation

This paper charts the evolution of an intercontinental Grid—INWA—from its first operation connecting Australia and Scotland; its subsequent extension to China; and its use to demonstrate the first large-scale research and education network for the Asia-Pacific region. The paper focuses on the gap between e-Science and e-Social Science arguing that the Grid topology is more compatible with the socio-legal demands of large-scale study of society than more dynamically distributed approaches, such as Cloud Computing. Foundational texts on Grid Computing and its appropriation by research programmes in the UK, USA and China have helped create a positive, symbolic value for Grid Computing. For INWA, this value helped when communicating the aims of the project to potential collaborators and so created the conditions for high-quality, socio-economic data to be placed in a collaborative, analytical environment. There is no equivalent symbolic value for Cloud Computing with potential consequences for its usefulness in establishing such collaborations in future.     

一种基于Grid的多媒体内容分布框架

基于互联网的大规模多媒体内容分发系统中,广泛地采用了数据复制技术来提高系统的性能。网格技术在构建大规模分布式信息系统方面,有着广阔的前景。本文提出了一种基于网格的多媒体内容分布框架,在网格基础服务的基础上,可以实现适用于流媒体应用的副本管理、传输和查找定位机制。此外,本文还对广域分布式信息系统中,副本的放置算法和用户请求调度机制等关键技术进行了深入探讨。     

A Science Driven Production Cyberinfrastructure��the Open Science Grid

This article describes the Open Science Grid, a large distributed computational infrastructure in the United States which supports many different high-throughput scientific applications, and partners (federates) with other infrastructures nationally and internationally to form multi-domain integrated distributed systems for science. The Open Science Grid consortium not only provides services and software to an increasingly diverse set of scientific communities, but also fosters a collaborative team of practitioners and researchers who use, support and advance the state of the art in large-scale distributed computing. The scale of the infrastructure can be expressed by the daily throughput of around seven hundred thousand jobs, just under a million hours of computing, a million file transfers, and half a petabyte of data movement. In this paper we introduce and reflect on some of the OSG capabilities, usage and activities.     

基于Web Service的分布式并行体绘制研究

目前,随着可视化要处理的数据量变得越来越大以及可视化操作变得越来越复杂,基于网格或Web服务的分布式可视化技术得到了快速发展.在体绘制领域,体数据的大规模增长,迫切期待提高计算设备的处理能力.基于Web服务的分布式技术的快速发展为体绘制的研究提供了新的途径.为了便于Web服务技术在体绘制中的应用的,该文提出了一种基于W...     

Two Ways to Grid: The Contribution of Open Grid Services Architecture (OGSA) Mechanisms to Service-Centric and Resource-Centric Lifecycles

Service Oriented Architectures (SOAs) support service lifecycle tasks, including Development, Deployment, Discovery and Use. We observe that there are two disparate ways to use Grid SOAs such as the Open Grid Services Architecture (OGSA) as exemplified in the Globus Toolkit (GT3/4). One is a traditional enterprise SOA use where end-user services are developed, deployed and resourced behind firewalls, for use by external consumers: a service-centric (or ‘first-order’) approach. The other supports end-user development, deployment, and resourcing of applications across organizations via the use of execution and resource management services: A Resource-centric (or ‘second-order’) approach. We analyze and compare the two approaches using a combination of empirical experiments and an architectural evaluation methodology (scenario, mechanism, and quality attributes) to reveal common and distinct strengths and weaknesses. The impact of potential improvements (which are likely to be manifested by GT4) is estimated, and opportunities for alternative architectures and technologies explored. We conclude by investigating if the two approaches can be converged or combined, and if they are compatible on shared resources.     

Studying protein folding on the Grid: experiences using CHARMM on NPACI resources under Legion

One benefit of a computational Grid is the ability to run high‐performance applications over distributed resources simply and securely. We demonstrated this benefit with an experiment in which we studied the protein‐folding process with the CHARMM molecular simulation package over a Grid managed by Legion, a Grid operating system. High‐performance applications can take advantage of Grid resources if the Grid operating system provides both low‐level functionality as well as high‐level services. We describe the nature of services provided by Legion for high‐performance applications. Our experiences indicate that human factors continue to play a crucial role in the configuration of Grid resources, underlying resources can be problematic, Grid services must tolerate underlying problems or inform the user, and high‐level services must continue to evolve to meet user requirements. Our experiment not only helped a scientist perform an important study, but also showed the viability of an integrated approach such as Legion's for managing a Grid. Copyright © 2004 John Wiley & Sons, Ltd.     

面向知识网格的本体学习研究

网格计算正在从单纯的面向大型计算的分布式资源共享发展为一种面向服务的架构,以实现透明而可靠的分布式系统集成。网格智能是指如何获取、预处理、表示和集成不同层次的网格服务(如HTML/XML/RDF/OWL文档、服务响应时间和服务质量等)的数据和信息,并最终转换为有用的智能(知识)。因为高层知识将在未来的网格应用起到越来越重要的作用,本体是知识网格实现的关键。文章提出了一种实现从Web文档中本体(半)自动构建的本体学习框架WebOntLearn,并讨论了本体学习中领域概念的抽取、概念之间关系的抽取和分类体系的自动构建等关键技术。     

On the Easy Use of Scientific Computing Services for Large Scale Linear Algebra and Parallel Decision Making with the P-Grade Portal

Scientific research is becoming increasingly dependent on the large-scale analysis of data using distributed computing infrastructures (Grid, cloud, GPU, etc.). Scientific computing (Petitet et al. 1999) aims at constructing mathematical models and numerical solution techniques for solving problems arising in science and engineering. In this paper, we describe the services of an integrated portal based on the P-Grade (Parallel Grid Run-time and Application Development Environment) portal (http://www.p-grade.hu) that enables the solution of large-scale linear systems of equations using direct solvers, makes easier the use of parallel block iterative algorithm and provides an interface for parallel decision making algorithms. The ultimate goal is to develop a single sign on integrated multi-service environment providing an easy access to different kind of mathematical calculations and algorithms to be performed on hybrid distributed computing infrastructures combining the benefits of large clusters, Grid or cloud, when needed.