面向生物信息的网格工作流开发与运行环境

网格工作流系统通过组合网格计算结点上的资源为生物计算等领域的复杂计算任务提供了支持。为了解决已有系统中用户需要对流程语言细节有较深入了解以及操作和配置较为复杂等问题,设计和实现了面向生物信息领域的网格工作流开发与运行环境。生物信息领域的用户使用该系统,可以通过web的方式实现对生物计算模型较高抽象层次的在线流程设计、流程作业的提交、作业状态的管理以及计算结果的查看等操作。最后通过生物计算领域的基因计算任务说明了系统对生物信息学应用的支撑功能。     

Grid Service Orchestration Using the Business Process Execution Language (BPEL)

Modern scientific applications often need to be distributed across Grids. Increasingly applications rely on services, such as job submission, data transfer or data portal services. We refer to such services as Grid services. While the invocation of Grid services could be hard coded in theory, scientific users want to orchestrate service invocations more flexibly. In enterprise applications, the orchestration of web services is achieved using emerging orchestration standards, most notably the Business Process Execution Language (BPEL). We describe our experience in orchestrating scientific workflows using BPEL. We have gained this experience during an extensive case study that orchestrates Grid services for the automation of a polymorph prediction application. Using this example, we explain the extent with which the BPEL language supports the definition of scientific workflows. We then describe the reliability, performance and scalability that can be achieved by executing a complex scientific workflow with ActiveBPEL, an industrial strength but freely available BPEL engine. *The work has been funded by the UK EPSRC through grants GR/R97207/01 (e-Materials) and GR/S90843/01 (OMII Managed Programme).     

Virtual workflow system for distributed collaborative scientific applications on Grids

Grid computing has become an effective computing technique in recent years. This paper develops a virtual workflow system to construct distributed collaborative applications for Grid users. The virtual workflow system consists three levels: abstract workflow system, translator and concrete workflow system. The research highlight of the implementation is that this workflow system is developed based on CORBA and Unicore Grid middleware. Furthermore, this implementation can support legacy application developed with Parco and C++ codes. This virtual workflow system can provide efficient GUI for users to organize distributed scientific collaborative applications and execute them on Grid resources. We present the design, implementation, and evaluation of this virtual workflow system in the paper.     

网格环境下费用约束的科学工作流可靠调度算法

网格基础设施是目前科学工作流应用规划、部署和执行的主要支撑环境.然而由于网格资源的自治、动态及异构性,如何在保障用户QoS约束下有效调度科学工作流是一个研究热点.针对费用约束下的科学工作流调度问题,为了提高其执行的可靠性,本文使用随机服务模型描述资源节点的动态服务能力并考虑本地任务负载对资源执行性能的影响,给出一种资源可靠性的评估方法,在此基础上提出一种费用约束下的科学工作流可靠调度算法RSASW.仿真实验结果表明RSASW算法相对于GAIN3,GreedyTime-CD及PFAS算法,对工作流的执行具有很好的可靠性保障.     

Practical Experience from Porting and Executing the Wien2k Application on the EGEE Production Grid Infrastructure

While the Grid promises to deliver a large number of computation nodes to a user, this computation power is not usable without the proper adaption of the application for the Grid. In this paper, we describe the methods used to port and execute a particular application, Wien2k, on the EGEE production Grid. First, the process of porting the application is described. Then, we investigate the measures necessary to execute the application in this production Grid environment efficiently. Although the focus is on this special application, we describe generic methods which can be applied to all applications. We specifically address: Creating a workflow from an application and mapping this workflow to a Grid workflow using the activity attraction pattern. We discuss workflow engines which support cycles in their application workflow. We investigate naïve and worker scheduling techniques. A short introduction into licensing on the Grid is given. Optimisation techniques such as deployment re-use are discussed. Different data transfer mechanisms, centralised data transfer, data re-use, storage element data transfer, and peer-to-peer data transfer are compared. The paper is concluded with suggestions for further workflow porting.     

Adaptive service scheduling for workflow applications in Service-Oriented Grid

When the workflow application is executed in Service-Oriented Grid (SOG), performance issues such as service scheduling should be considered, to achieve high and stable performance in execution. However, most of the prior works on workflow management neither study the performance issues nor provide evaluation methodologies on the performance of Grid Services. Therefore, it is infeasible to apply for the service scheduling problem in SOG. In this paper, we propose and model evaluation metrics for the Grid Service performance. The metrics are extracted based on common properties of Grid Services and are used to quantify and evaluate the performance of an individual Grid Service. With these metrics, we develop a service scheduling scheme with a list scheduling heuristic, to choose proper and optimal Grid Services for tasks in workflow applications. It ensures high performance in the execution of the workflow applications. In addition, we propose a low-overhead rescheduling method, referred to as Adaptive List Scheduling for Service (ALSS), to adapt to the dynamic nature of a grid environment. ALSS provides stable performance for workflow applications, even in abnormal circumstances. Finally, we design an experimental environment with actual traces and perform simulations to quantify the benefits of our approach. Throughout the experiments, we demonstrate that ALSS outperforms conventional scheduling methods. Our scheme produces a scheduling performance that is superior to AHEFT by 50.2%, SLACK by 50.8%, HEFT by 68.3%, MaxMin by 72.0%, MinMin by 71.0%, and Myopic by 69.8%.     

基于动态有色Petri网的网格服务工作流模型的研究

在深入了解网格技术、网格服务和网格工作流的概念、特点及其应用的基础上,提出了一种可行的网格服务工作流系统模型,重点介绍了动态优化建模技术、动态调度算法的实现思想.定义了一种动态有色Petri网作为服务工作流的建模工具,支持服务工作流的动态优化建模和动态调度,并为服务工作流模型提供性能评价依据.验证表明采用该模型能够很好地满足用户的QoS要求,并且有助于提高资源利用率.     

Dependable Grid Workflow Scheduling Based on Resource Availability

Due to the highly dynamic feature, dependable workflow scheduling is critical in the Grid environment. Various scheduling algorithms have been proposed, but seldom consider the resource reliability. Current Grid systems mainly exploit fault tolerance mechanism to guarantee the dependable workflow execution, which, however, wastes system resources. The paper proposes a dependable Grid workflow scheduling system (called DGWS). It introduces a Markov Chain-based resource availability prediction model. Based on the model, a reliability cost driven workflow scheduling algorithm is presented. The performance evaluation results, including the simulation on both parametric randomly generated DAGs and two real scientific workflow applications, demonstrate that compared to present workflow scheduling algorithms, DGWS improves the success ratio of tasks and diminishes the makespan of workflow, so improves the dependability of workflow execution in the dynamic Grid environments.     

Petri net based Grid workflow verification and optimization

Graphical workflow modeling tools, such as UML and DAG, can facilitate users to express workflow process logic, but lack of abilities to carry out simulation and correctness checking. In this paper, based on Petri net, we propose a service composition oriented Grid workflow model and its related six elementary workflow patterns: sequence, condition, iteration, concurrency, synchronization, and triggering. In addition, we present our Grid workflow analysis approaches on three aspects: workflow reachability verification, workflow deadlock verification, and workflow optimization. The experimental results show that our workflow verification and optimization mechanisms are feasible and efficient.     

Campus Grids Meet Applications: Modeling, Metascheduling and Integration

Air Quality Forecasting (AQF) is a new discipline that attempts to reliably predict atmospheric pollution. An AQF application has complex workflows and in order to produce timely and reliable forecast results, each execution requires access to diverse and distributed computational and storage resources. Deploying AQF on Grids is one option to satisfy such needs, but requires the related Grid middleware to support automated workflow scheduling and execution on Grid resources. In this paper, we analyze the challenges in deploying an AQF application in a campus Grid environment and present our current efforts to develop a general solution for Grid-enabling scientific workflow applications in the GRACCE project. In GRACCE, an application’s workflow is described using GAMDL, a powerful dataflow language for describing application logic. The GRACCE metascheduling architecture provides the functionalities required for co-allocating Grid resources for workflow tasks, scheduling the workflows and monitoring their execution. By providing an integrated framework for modeling and metascheduling scientific workflow applications on Grid resources, we make it easy to build a customized environment with end-to-end support for application Grid deployment, from the management of an application and its dataset, to the automatic execution and analysis of its results.The work has been performed as part of the University of Houston’s Sun Microsystems Center of Excellence in Geosciences [38].     

P-GRADE: A Grid Programming Environment

P-GRADE provides a high-level graphical environment to develop parallel applications transparently both for parallel systems and the Grid. P-GRADE supports the interactive execution of parallel programs as well as the creation of a Condor, Condor-G or Globus job to execute parallel programs in the Grid. In P-GRADE, the user can generate either PVM or MPI code according to the underlying Grid where the parallel application should be executed. PVM applications generated by P-GRADE can migrate between different Grid sites and as a result P-GRADE guarantees reliable, fault-tolerant parallel program execution in the Grid. The GRM/PROVE performance monitoring and visualisation toolset has been extended towards the Grid and connected to a general Grid monitor (Mercury) developed in the EU GridLab project. Using the Mercury/GRM/PROVE Grid application monitoring infrastructure any parallel application launched by P-GRADE can be remotely monitored and analysed at run time even if the application migrates among Grid sites. P-GRADE supports workflow definition and co-ordinated multi-job execution for the Grid. Such workflow management can provide parallel execution at both inter-job and intra-job level. Automatic checkpoint mechanism for parallel programs supports the migration of parallel jobs inside the workflow providing a fault-tolerant workflow execution mechanism. The paper describes all of these features of P-GRADE and their implementation concepts.     

基于服务质量的网格工作流系统研究

网格工作流可以定义成为完成特定目标而在异构和分布的资源上以一定次序执行的网格应用服务的集合.在网格工作流中引入服务质量,为网格服务的调度提供了依据,使得工作流的执行更加满足用户需求.通过使用服务质量可以对网格工作流进行选择和执行,从而更加符合用户的期望.提出了一种典型的基于服务质量的网格工作流管理体系结构及相关的服务质量参数体系,讨论了基于服务质量的工作流调度算法,并给出一个基于快速遗传算法的应用实例.     

基于网格服务的电子商务自适应工作流模型

网格服务是分布式应用的新技术，能够为电子商务的工作流框架设计和工作流管理提供更为先进的技术支持。基于网格服务的工作流管理系统具有较好的自适应性、分布性和动态性。自适应工作流框架（AWFF）是一个分层结构，将工作流模型分为功能层和实例层。功能层定义了整个工作流的基础功能模型，实例层以事件触发机制为支撑，可根据动态的网格环境和多变的客户需求，通过网格服务实现进行自适应。将自适应工作流框架应用到网络电子商务系统中，能够更好地适应分布的、动态环境，提高整个流程的可控性和管理效率。     

网格工作流中基于商品市场的服务选择

服务选择是网格工作流的重要环节。针对现有Web服务组合方法对服务质量（QoS）贪婪索取而不利于网格资源有效利用的问题,提出基于商品市场的"按需"网格服务选择方法：根据面向工作流全局的评价模型,以服务使用者需求为约束条件,将服务选择建模为0-1多维背包问题;并引入经济学中的"价格-供需关系"改进了服务选择模型。仿真实验表明：基于商品市场的服务选择均衡了系统负载,有利于网格资源的高效利用。     

A Taxonomy of Workflow Management Systems for Grid Computing

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. We also survey several representative Grid workflow systems developed by various projects world-wide to demonstrate the comprehensiveness of the taxonomy. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.     

一种基于OGSI的网格工作流描述语言

网格被称为下一代的Internet。网格工作流平台是实现网格计算的重要基础软件,而网格工作流描述语言则是定义和使用工作流的重要工具。该文提出了一种基于OGSI规范的网格工作流描述语言GBPEL。     

服务质量感知的网格工作流调度

在网格工作流中引入服务质量,可以使网格中的资源更好地围绕用户的要求进行组织和分配,服务质量为工作流执行过程中选择成员服务提供了依据.工作流服务质量的估算和服务质量感知的工作流调度是实现服务质量感知的网格工作流的两个关键问题.基于一种网格工作流模型讨论了网格工作流的服务质量参数体系,提出了工作流服务质量的估算算法和网格工作流调度数学模型,并提出了基于遗传算法的调度方法.仿真实验表明,该调度算法具有较好的收敛性.     

Performance metrics and ontologies for Grid workflows

Many Grid workflow middleware services require knowledge about the performance behavior of Grid applications/services in order to effectively select, compose, and execute workflows in dynamic and complex Grid systems. To provide performance information for building such knowledge, Grid workflow performance tools have to select, measure, and analyze various performance metrics of workflows. However, there is a lack of a comprehensive study of performance metrics which can be used to evaluate the performance of a workflow executed in the Grid. Moreover, given the complexity of both Grid systems and workflows, semantics of essential performance-related concepts and relationships, and associated performance data in Grid workflows should be well described. In this paper, we analyze performance metrics that performance monitoring and analysis tools should provide during the evaluation of the performance of Grid workflows. Performance metrics are associated with multiple levels of abstraction. We introduce an ontology for describing performance data of Grid workflows and illustrate how the ontology can be utilized for monitoring and analyzing the performance of Grid workflows.     

Optimizing Grid-Based Workflow Execution

Large-scale applications can be expressed as a set of tasks with data dependencies between them, also known as application workflows. Due to the scale and data processing requirements of these applications, they require Grid computing and storage resources. So far, the focus has been on developing easy to use interfaces for composing these workflows and finding an optimal mapping of tasks in the workflow to the Grid resources in order to minimize the completion time of the application. After this mapping is done, a workflow execution engine is required to run the workflow over the mapped resources. In this paper, we show that the performance of the workflow execution engine in executing the workflow can also be a critical factor in determining the workflow completion time. Using Condor as the workflow execution engine, we examine the various factors that affect the completion time of a fine granularity astronomy workflow. We show that changing the system parameters that influence these factors and restructuring the workflow can drastically reduce the completion time of this class of workflows. We also examine the effect on the optimizations developed for the astronomy application on a coarser granularity biology application. We were able to reduce the completion time of the Montage and the Tomography application workflows by 90% and 50%, respectively.     

Workflow-Based Data Parallel Applications on the EGEE Production Grid Infrastructure

Setting up and deploying complex applications on a Grid infrastructure is still challenging and the programming models are rapidly evolving. Efficiently exploiting Grid parallelism is often not straight forward. In this paper, we report on the techniques used for deploying applications on the EGEE production Grid through four experiments coming from completely different scientific areas: nuclear fusion, astrophysics and medical imaging. These applications have in common the need for manipulating huge amounts of data and all are computationally intensive. All the cases studied show that the deployment of data intensive applications require the development of more or less elaborated application-level workload management systems on top of the gLite middleware to efficiently exploit the EGEE Grid resources. In particular, the adoption of high level workflow management systems eases the integration of large scale applications while exploiting Grid parallelism transparently. Different approaches for scientific workflow management are discussed. The MOTEUR workflow manager strategy to efficiently deal with complex data flows is more particularly detailed. Without requiring specific application development, it leads to very significant speed-ups.