Analysis and Provision of QoS for Distributed Grid Applications

Grid computing provides the infrastructure necessary to access and use distributed resources as part of virtual organizations. When used in this way, Grid computing makes it possible for users to participate in collaborative and distributed applications such as tele-immersion, visualization, and computational simulation. Some of these applications operate in a collaborative mode, requiring data to be stored and delivered in a timely manner. This class of applications must adhere to stringent real-time constraints and Quality-of-Service (QoS) requirements. A QoS management approach is therefore required to orchestrate and guarantee the timely interaction between such applications and services. We discuss the design and a prototype implementation of a QoS system, and demonstrate how we enable Grid applications to become QoS compliant. We validate this approach through a case study of an image processing task derived from a nanoscale structures application.     

一种多用户MapReduce集群的作业调度算法的设计与实现

随着更多的企业开始使用数据密集型集群计算系统如Hadoop和Dryad实现了更多的应用,多用户间共享MapRe-duce集群这种既减少了建立独立集群的代价,同时又使得多用户间可以共享更多的大数据集资源的需求日益增多。在公平调度算法的基础上,结合槽分配延迟和优先级的技术,本文提出了一种改进算法,可以实现更好的数据本地性,改善整个系统的计算性能如吞吐率、响应时间等;同时为了满足差别化的商业服务,通过对用户设置相应的优先级保证紧急任务的完成。     

中国气象局Internet接入系统整改设计

优化中国气象局现有Internet接入系统,增强Internet接入系统的稳定性、安全性。利用负载均衡对多线路资源进行整合与优化,区别对待办公用户、科研用户、临时用户,更好的为不同类型的用户服务。对办公用户上互联网,利用流量管理技术限制消耗带宽、影响工作效率、违法乱纪的行为;对办公区用户上网、VPN回访行为进行管理和审计,加强对用户使用互联网的内部监管。保障业务系统的互联网资源,提高对外网服务器区的安全性,依照等级保护的要求对互联网DMZ区进行划分,利用入侵防御、防DDoS、网络审计、漏洞扫描等安全技术构建统一的网络安全防御体系,为不同等级的信息系统提供分级别的网络安全服务。     

Enhancing grid security by fine-grained behavioral control and negotiation-based authorization

Nowadays, Grid has become a leading technology in distributed computing. Grid poses a seamless sharing of heterogeneous computational resources belonging to different domains and conducts efficient collaborations between Grid users. The core Grid functionality defines computational services which allocate computational resources and execute applications submitted by Grid users. The vast models of collaborations and openness of Grid system require a secure, scalable, flexible and expressive authorization model to protect these computational services and Grid resources. Most of the existing authorization models for Grid have granularity to manage access to service invocations while behavioral monitoring of applications executed by these services remains a responsibility of a resource provider. The resource provider executes an application under a local account, and acknowledges all permissions granted to this account to the application. Such approach poses serious security threats to breach system functionality since applications submitted by users could be malicious. We propose a flexible and expressive policy-driven credential-based authorization system to protect Grid computational services against a malicious behavior of applications submitted for the execution. We split an authorization process into two levels: a coarse-grained level that manages access to a computational service; and a fine-grained level that monitors the behavior of applications executed by the computational service. Our framework guarantees that users authorized on a coarse-grained level behave as expected on the fine-grained level. Credentials obtained on the coarse-grained level reflect on fine-grained access decisions. The framework defines trust negotiations on coarse-grained level to overcome scalability problem, and preserves privacy of credentials and security policies of, both, Grid users and providers. Our authorization system was implemented to control access to the Globus Computational GRAM service. A comprehensive performance evaluation shows the practical scope of the proposed system.

一种可扩展的气象水文网格模型

基于气象水文和网格技术的发展现状,结合气象水文领域的应用特点,提出了一种可扩展的气象水文网格模型,重点研究网格技术在气象水文领域的应用框架及其模块结构。该网格模型首先通过网格技术将异构的气象水文资源进行整合和统一管理;抽取各气象水文应用的公共元素,提供统一的用户认证和用户活动控制;并在此基础上通过动态承载气象水文应用服务的办法来保证各类应用的独立运行,同时各种新的气象水文应用可以加载到已有的网格环境。分析表明该气象水文网格模型具有很强的灵活性和可扩展性。     

An optimization-based adaptive resource management framework for economic Grids: A switching mechanism

The application of Grid computing has been broadening day by day. An increasing number of users has led to the requirement of a job scheduling process, which can benefit them through optimizing their utility functions. On the other hand, resource providers are exploring strategies suitable for economically efficient resource allocation so that they can maximize their profit through satisfying more users. In such a scenario, economic-based resource management strategies (economic models) have been found to be compelling to satisfy both communities. However, existing research has identified that different economic models are suitable for different scenarios in Grid computing. The Grid application and resource models are typically very dynamic, making it challenging for a particular model for delivering stable performance all the time. In this work, our focus is to develop an adaptive resource management architecture capable of dealing with multiple models based on the models’ domains of strengths (DOS). Our preliminary results show promising outcomes if we consider multiple models rather than relying on a single model throughout the life cycle of a Grid.     

Reliability of grid service systems

Grid computing system is different from conventional distributed computing systems by its focus on large-scale resource sharing, where remote accessing and information communication have great influence on grid computing. As an important metric for services, the grid service reliability is studied in this paper. There are some earlier studies on the reliability analysis of conventional small-scale distributed systems, which ignored the communication time and processing time when studying the reliability, but it is not practical in the reliability analysis of grid services due to the wide-area property of the network. Thus, this paper presents a model for the grid services and develops an approach to analyze the grid service reliability. Detailed algorithms to compute the grid service reliability are presented and a self-sensing technology is proposed for parameterization and monitoring. A numerical example is used for the purpose of illustration. Finally, the new model is compared with different types of conventional models, through which this model is verified more suitable for grid service reliability.     

A Grid‐enabled problem‐solving environment for advanced reservoir uncertainty analysis

Uncertainty analysis is critical for conducting reservoir performance prediction. However, it is challenging because it relies on (1) massive modeling‐related, geographically distributed, terabyte, or even petabyte scale data sets (geoscience and engineering data), (2) needs to rapidly perform hundreds or thousands of flow simulations, being identical runs with different models calculating the impacts of various uncertainty factors, (3) an integrated, secure, and easy‐to‐use problem‐solving toolkit to assist uncertainty analysis. We leverage Grid computing technologies to address these challenges. We design and implement an integrated problem‐solving environment ResGrid to effectively improve reservoir uncertainty analysis. The ResGrid consists of data management, execution management, and a Grid portal. Data Grid tools, such as metadata, replica, and transfer services, are used to meet massive size and geographically distributed characteristics of data sets. Workflow, task farming, and resource allocation are used to support large‐scale computation. A Grid portal integrates the data management and the computation solution into a unified easy‐to‐use interface, enabling reservoir engineers to specify uncertainty factors of interest and perform large‐scale reservoir studies through a web browser. The ResGrid has been used in petroleum engineering. Copyright © 2008 John Wiley & Sons, Ltd.     

基于网格服务的校园信息共享系统的研究

随着信息和网络技术的发展,信息化建设已成为高校的一项重要的基础工作。然而在建信息系统时。缺少全局的规划,造成了学校各部门信息系统异构,难以共享,形成了“信息孤岛”。为了消除校园中的“信息孤岛”,本文提出了一个基于网格服务信息共享系统的方案,将校园内信息系统进行整合,实现信息共享。     

基于Web Service的计算网格入口的设计与实现

在对网格系统进行分析的基础上,设计了一种三层结构的网格入口体系,采用WebService技术实现了基于西安交通大学高性能计算环境的计算网格入口,并对系统的特点进行了分析评价。入口提供了友好的基于浏览器的用户使用界面,并且屏蔽了环境的异构性和松耦合性,极大地方便了用户对于网格资源的使用。     

Data discovery algorithm for scientific data grid environment

In modern scientific computing communities, scientists are involved in managing massive amounts of very large data collections in a geographically distributed environment. Research in the area of grid computing has given us various ideas and solutions to address these requirements. Data grid mostly deals with large computational problems and provides geographically distributed resources for large-scale data-intensive applications that generate large data sets. Peer-to-peer (P2P) networks have also become a major research topic over the last few years. In a distributed P2P system, a discovery algorithm is required to locate specific information, applications, or users within the system. In this research work, we present our scientific data grid as a large P2P-based distributed system model. By using this model, we study various discovery algorithms for locating data sets in a data grid system. The algorithms we studied are based on the P2P architecture. We investigate these algorithms using our Grid Simulator developed using PARSEC. In this paper, we illustrate our scientific data grid model and our Grid Simulator. We then analyze the performance of the discovery algorithms relative to their average number of hop, success rates and bandwidth consumption.     

面向服务的学习资源网格及其演示系统设计与实现

随着研究的发展,网格技术由计算为中心演变为以服务为中心。文章以学习资源为研究对象构建学习资源网格,将性能各异的学习资源抽象设计为网格服务,屏蔽底层资源的异构性。在此基础上对学习资源网格进行了模拟演示,将抽象的网格技术可视化,形象地展示学习资源网格的工作原理、特点及服务调用流程,使网格研究者和用户对学习资源网格有更清晰直观地理解和认识。     

Transparent access to Grid resources for user software

Grid computing promises access to large amounts of computing power, but so far adoption of Grid computing has been limited to highly specialized experts for three reasons. First, users are used to batch systems, and interfaces to Grid software are often complex and different to those in batch systems. Second, users are used to having transparent file access, which Grid software does not conveniently provide. Third, efforts to achieve wide‐spread coordination of computers while solving the first two problems is hampered when clusters are on private networks. Here we bring together a variety of software that allows users to almost transparently use Grid resources as if they were local resources while providing transparent access to files, even when private networks intervene. As a motivating example, the BaBar Monte Carlo production system is deployed on a truly distributed environment, the European DataGrid, without any modification to the application itself. Copyright © 2005 John Wiley & Sons, Ltd.     

Quality of Service Negotiation for Commercial Medical Grid Services

The GEMSS project has developed a service-oriented Grid that supports the provision of medical simulation services by service providers to clients such as hospitals. We outline the GEMSS architecture, legal framework and the security features that characterise the GEMSS infrastructure. High levels of quality of service are required and we describe a reservation-based approach to quality of service, employing a quality of service management system that iteratively finds suitable reservations and uses application specific performance models. The GEMSS Grid is a commercial environment so we support flexible pricing models and a FIPA reverse English auction protocol. Signed Web Service Level Agreement contracts are exchanged to commit parties to a quality of service agreement before job execution occurs. We run four experiments across European countries using high performance computing resources running advanced resource reservation schedulers. These experiments provide evidence for our Grid’s rational behaviour, both at the level of service provider quality of service management and at the higher level of the client choosing between competing service providers. The results lend support to our economic model and the technology we use for our medical application domain.     

An Interoperable Grid Information System for Integrated Resource Monitoring Based on Virtual Organizations

In many Grid infrastructures different kinds of information services are in use, which utilize different incompatible data structures and interfaces to encode and provide their data. Homogeneous monitoring of these infrastructures with the monitoring data being accessible everywhere independently of the middleware which provided it, is the basis for a consistent status reporting on the Grids’ resources and services. Thus, interoperability or interoperation between the different information services in a heterogeneous Grid infrastructure is required. Monitoring data must contain the identity of the affected Virtual Organization (VO) so that it can be related to the resources and services the VO has allocated to enable VO-specific information provision. This paper describes a distributed architecture for an interoperable information service, which combines data unification and categorization with policies for VO membership, VO resource management and data transformations. This service builds the basis for an integrated and interoperating monitoring of Grids, which provide their data to more than one VO and utilize heterogeneous information services.     

基于经济模型的资源代理

网格计算是为解决大规模资源密集型问题而提出的新一代计算平台,是当前并行和分布处理技术的一个发展方向,而资源管理是计算网格的关键技术之一。对各种各样可利用资源的整合和管理是网格应用的基础,而资源的分布性、动态性、异构性、自治性和需要协调一致性使得网格资源的管理调度成为一个棘手的问题。目前基于市场的经济资源管理和调度算法非常适合计算网格中的资源管理问题,但有调度价格不能更改、负载平衡等问题。文中提出了“网格环境下基于经济模型的资源代理”,依靠多维QoS指导的调度策略和经济模型的启发式调节资源价格,改进和优化计算网格资源的分配。     

A collaborative virtual geographic environment based on P2P and Grid technologies

Solving a geographic problem usually requires collaborative work among a group of people in different geographic locations. Collaborative virtual geographic environment (CVGE), an integrated technology, offers an intuitive, efficient, and interactive visualization environment through which geographically separated users can explore complicated spatial information and conduct collaborative work. In this paper, two new technologies, Peer-to-Peer (P2P) and Grid computing, are tightly coupled to develop a CVGE system. This paper evaluates the potential contributions of the P2P and Grid technology to CVGE systems. Using a Grid based system architecture efficiently integrates and shares geographically distributed resources as well as modeling procedures built on different platforms. To offer a shared and interactive virtual collaborative geographic environment for resolving geographic problems, we developed several P2P services including a terrain visualization collaboration service and a video collaboration service. Finally, a CVGE prototype system is implemented for collaboration on silt dam planning on the Loess plateau. The experimental results show that the scheme developed in this paper is efficient and feasible.     

基于WebGIS信息整合与利用的林业电子政务

林业电子政务要向社会公众提供专业服务。面向服务的架构和万维网地理信息系统是处理林业电子政务信息资源整合和利用的重要技术工具。文章以林业电子政务中森林火点检测的应用流程为例展示上述技术的运用方法。工作流程可以利用分布式的遥感数据、矢量辅助数据资源、森林火点检测算法和万维网地图发布服务,通过一个网格的基础架构把这些资源集成起来形成林业专业应用。文章展示的试验床用Web服务技术封装了专业算法。网格中间件采用的是自主研制的SIRGE,调度资源的工作流中间件采用的是自主研制的SIGApp,它们的专长是可以处理空间数据。用于通过这个试验床可以访问基于不同来源数据和服务得到的森林火点信息,为林业电子政务服务。     

A Framework for Trust and Reputation in Grid Environments

This paper examines the role of trust and reputation in Grid environments. As Grid technology becomes standardized and stable, business models are invented and increasingly applied, and economic implications can be observed. Asymmetrically distributed information may allow for opportunistic behavior of service providers or users who opportunistically exploit the information gap between providers and consumers on the quality of services. The paper takes up these economic issues by proposing a reputation-based conceptual framework for enabling future open Grid markets, to recommend the most promising Grid architecture and a corresponding reputation approach in a particular case.     

Modeling and analysis of the effects of QoS and reliability on pricing, profitability, and risk management in multiperiod grid-computing networks

In this paper we develop a network equilibrium model for optimal pricing and resource allocation in Computational Grid Network. We consider a general network economy model with Grid Resource Providers, Grid Resource Brokers and Grid Users. The proposed framework allows for the modeling and theoretical analysis of Computational Grid Markets that considers a non-cooperative behavior of decision-makers in the same tier of the grid computing network (such as, for example, Grid Resource Providers) as well as cooperative behavior between tiers (between Resource Providers and Grid Brokers). We introduce risk management into the decision making process by analyzing the decision-marker's reliability and quality of service (QoS) requirement. We analyze resource allocation patterns as well as equilibrium price based on demand, supply, and cost structure of the grid computing market network. We specifically answer the following questions with several numerical examples: How do system reliability levels affect the QoS levels of the service providers and brokers under competition? How do system reliability levels affect the profits of resource providers and brokers in a competitive market? How do system reliability levels influence the pricing of the services in a competitive environment? How do users' service request types, QoS requirements, and timing concerns affect users' behaviors, costs and risks in equilibrium? How does the market mechanism allocate resources to satisfy the demands of users? We find that for users who request same services certain timing flexibility can not only reduce the costs but also lower the risks. The results indicated that the value of QoS can be efficiently priced based on the heterogeneous service demands.