A collaborative scheduling approach for service-driven scientific workflow execution

Scientific workflow execution often spans multiple self-managing administrative domains to obtain specific processing capabilities. Existing (global) analysis techniques tend to mandate every domain-specific application to unveil all private behaviors for scientific collaboration. In practice, it is infeasible for a domain-specific application to disclose its process details (as a private workflow fragment) for privacy or security reasons. Consequently, it is a challenging endeavor to coordinate scientific workflows and its distributed domain-specific applications. To address this problem, we propose a collaborative scheduling approach that can deal with temporal dependencies between a scientific workflow and a private workflow fragment. Under this collaborative scheduling approach, a private workflow fragment could maintain the temporal consistency with a scientific workflow in resource sharing and task enactments. Further, an evaluation is also presented to demonstrate the proposed approach for coordinating multiple scientific workflow executions in a concurrent environment.     

Using a suite of ontologies for preserving workflow-centric research objects

Scientific workflows are a popular mechanism for specifying and automating data-driven in silico experiments. A significant aspect of their value lies in their potential to be reused. Once shared, workflows become useful building blocks that can be combined or modified for developing new experiments. However, previous studies have shown that storing workflow specifications alone is not sufficient to ensure that they can be successfully reused, without being able to understand what the workflows aim to achieve or to re-enact them. To gain an understanding of the workflow, and how it may be used and repurposed for their needs, scientists require access to additional resources such as annotations describing the workflow, datasets used and produced by the workflow, and provenance traces recording workflow executions.In this article, we present a novel approach to the preservation of scientific workflows through the application of research objects—aggregations of data and metadata that enrich the workflow specifications. Our approach is realised as a suite of ontologies that support the creation of workflow-centric research objects. Their design was guided by requirements elicited from previous empirical analyses of workflow decay and repair. The ontologies developed make use of and extend existing well known ontologies, namely the Object Reuse and Exchange (ORE) vocabulary, the Annotation Ontology (AO) and the W3C PROV ontology (PROVO). We illustrate the application of the ontologies for building Workflow Research Objects with a case-study that investigates Huntington’s disease, performed in collaboration with a team from the Leiden University Medial Centre (HG-LUMC). Finally we present a number of tools developed for creating and managing workflow-centric research objects.     

Information flow analysis of scientific workflows

Recently, scientific workflows have emerged as a platform for automating and accelerating data processing and data sharing in scientific communities. Many scientific workflows have been developed for collaborative research projects that involve a number of geographically distributed organizations. Sharing of data and computation across organizations in different administrative domains is essential in such a collaborative environment. Because of the competitive nature of scientific research, it is important to ensure that sensitive information in scientific workflows can be accessed by and propagated to only authorized parties. To address this problem, we present techniques for analyzing how information propagates in scientific workflows. We also present algorithms for incrementally analyzing how information propagates upon every change to an existing scientific workflow.     

Workflow View Driven Cross-Organizational Interoperability in a Web Service Environment

Workflow technology has recently been employed not only within businesses but also as a framework for implementing e-services over the Internet. Such e-services typically require collaborative enactment of workflows across multiple organizations. In this paper, we propose the use of workflow views as a fundamental support mechanism for the interoperability of multiple workflows across business organizations. We present a meta-model of workflow views and their semantics using a cross-organization workflow example based on a supply-chain e-service. We also formulate an interoperation model of workflow views and its consistency criteria. Finally, this paper presents an implementation of the model based on XML and contemporary Web services technologies, with adaptation to our E-ADOME workflow engine.     

跨组织工作流的动态协同技术研究

跨组织的工作流致力于跨越组织边界进行业务重组。针对多个组织间动态协同的复杂性,在组织间协作过程中必须考虑对组织的隐私信息、已建立的工作流和已建立的工作流管理系统的保护,解决成员间进行信息共享时出现的问题,从而真正实现协同商务。为满足这些需求,提出了基于视图的方法,这个方法考虑了工作流和资源的部分可见性,不同程度的可见性可以使企业保持其内部工作流隐私和安全性所需的水平,从而把工作流的可见性降低到合作所需的最低水平。研究的目的是使得组织在增强对外交互能力的同时,保护组织内部的敏感信息。     

A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds

In the last years, scientific workflows have emerged as a fundamental abstraction for structuring and executing scientific experiments in computational environments. Scientific workflows are becoming increasingly complex and more demanding in terms of computational resources, thus requiring the usage of parallel techniques and high performance computing (HPC) environments. Meanwhile, clouds have emerged as a new paradigm where resources are virtualized and provided on demand. By using clouds, scientists have expanded beyond single parallel computers to hundreds or even thousands of virtual machines. Although the initial focus of clouds was to provide high throughput computing, clouds are already being used to provide an HPC environment where elastic resources can be instantiated on demand during the course of a scientific workflow. However, this model also raises many open, yet important, challenges such as scheduling workflow activities. Scheduling parallel scientific workflows in the cloud is a very complex task since we have to take into account many different criteria and to explore the elasticity characteristic for optimizing workflow execution. In this paper, we introduce an adaptive scheduling heuristic for parallel execution of scientific workflows in the cloud that is based on three criteria: total execution time (makespan), reliability and financial cost. Besides scheduling workflow activities based on a 3-objective cost model, this approach also scales resources up and down according to the restrictions imposed by scientists before workflow execution. This tuning is based on provenance data captured and queried at runtime. We conducted a thorough validation of our approach using a real bioinformatics workflow. The experiments were performed in SciCumulus, a cloud workflow engine for managing scientific workflow execution.     

一种支持流程服务集成及协作的工作流描述语言PS-XPDL

在面向服务体系结构和Web服务技术快速发展的条件下，对封装成服务的业务过程集成和协作的支持成为工作流描述语言的发展趋势。通过对流程协作的层次模型和场景模型的分析，提出了一种基于WfMC工作流元模型的扩展元模型，并根据该元模型对XPDL进行了相关元素扩展,得到一种支持流程集成和协作的工作流语言PS-XPDL。该语言对流程服务的协作关系及数据同步具有完备、准确的表达，同时其基于执行支持层标准协议的扩展也使得流程服务的集成及协作易于实现。     

Inheritance of Interorganizational Workflows: How to Agree to Disagree Without Loosing Control?

Internet-based technology, E-commerce, and the rise of networked virtual enterprises have fueled the need for interorganizational workflows. Although XML allows trading partners to exchange information, it cannot be used to coordinate activities in different organizational entities. Business-to-business processes are hindered by the lack of a common language to support collaboration. This paper describes the P2P (Public-To-Private) approach which addresses one of the most notorious problems in this domain: How to design an interorganizational workflow such that there is local autonomy without compromising the consistency of the overall process. The approach uses a notion of inheritance and consists of three steps: (1) create a common understanding of the interorganizational workflow by specifying the shared public workflow, (2) partition the public workflow over the organizational entities involved, and (3) for each organizational entity: create a private workflow which is a subclass of the relevant part of the public workflow. This paper shows that this approach avoids typical anomalies in business-to-business collaboration (e.g., deadlocks and livelocks) and yields an interorganizational workflow which is guaranteed to realize the behavior specified in the public workflow.     

BPS工作流平台在协同办公系统中的应用

针对企业内部管理协作高效需求的日益凸显,介绍了普元工作流平台的参考模型。基于普元BPS工作流平台,构建了企业协同办公系统功能框架,并以企业核心业务发文处理流程为例,探讨了基于BPS工作流系统的流程设计过程,对构建信息共享和业务协同的高效企业具有重要意义。     

Exploring Workflow Interoperability for Neuroimage Analysis on the SHIWA Platform

Neuroimaging is a field that benefits from distributed computing infrastructures (DCIs) to perform data processing and analysis, which is often achieved using Grid workflow systems. Collaborative research in neuroimaging requires ways to facilitate exchange between different groups, in particular to enable sharing, re-use and interoperability of applications implemented as workflows. The SHIWA project provides solutions to facilitate sharing and exchange of workflows between workflow systems and DCI resources. In this paper we present and analyse how the SHIWA Platform was used to implement various cases in which workflow exchange supports collaboration in neuroscience. The SHIWA Platform and the implemented solutions are described and analysed from a “user” perspective, in this case workflow developers and neuroscientists. We conclude that the platform in its current form is valuable for these cases, and we identify remaining challenges.     

基于协作文档的异构工作流管理系统协同模型*

提出了一种基于协作文档的模型,以实现异构工作流管理系统之间的协同工作,讨论了其内部机理与模型建立的理由。并针对此模型,提出了一个有四层架构的协同代理（CA）,以XML文档为中间介质,通过协同代理隐藏异构工作流管理系统的内部细节,由此实现了一种灵活,轻量级的互连。可以广泛地应用于各种平台和环境中,实现计算机协同工作,提高效率和信息共享。     

Metadata management for distributed first principles calculations in VLab—A collaborative cyberinfrastructure for materials computation

This paper describes the metadata and metadata management algorithms necessary to handle the concurrent execution of multiple tasks from a single workflow, in a collaborative service oriented architecture environment. Metadata requirements are imposed by the distributed workflow that calculates thermoelastic properties of materials at high pressures and temperatures. The scientific relevance of this workflow is also discussed. We explain the basic metaphor, the receipt, underlying the metadata management. We show the actual java representation of the receipt, and explain how it is converted to XML in order to be transferred between servers, and stored in a database. We also discuss how the collaborative aspect of user activity on running workflows could potentially lead to race conditions, how this affects requirements on metadata, and how these race conditions are precluded. Finally we describe an additional metadata structure, complementary to the receipts, that contains general information about the workflow.     

A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids

Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.     

A policy enforcement framework for verification and control of service collaboration

This paper proposes a policy enforcement framework to dynamically verify and control the collaboration process in Service-Oriented Architecture (SOA). Dynamic SOA collaboration is different from traditional service collaboration where the workflow is pre-defined at design time while the services used can be discovered at runtime. In dynamic collaboration, both the workflows and services can be determined at runtime. As they will be determined at runtime, many verification activities can be performed at runtime. This paper proposes a dynamic policy enforcement framework that follows the dynamic SOA collaboration process to ensure that various system constraints can be enforced at runtime. The framework includes a policy specification language, a policy completeness and consistency checking, and distributed policy enforcement. Finally, this paper presents the construction of a sample example to illustrate these features with quantitative performance data.     

Contract-collaboration network method for modeling manufacturing resource control workflows

This paper presents contract-collaboration network (CC-Net) method that is developed to model manufacturing resource control workflows. The CC-Net is an object-oriented class diagram. It depicts the contract-collaboration relationships among the classes in a manufacturing system, with constraints. The CC-Net method uses a primitive modeling block called collaboration module by which the CC-Net is established systematically. This idea is very similar to that of the Lego® block toy. Unlike most workflow modeling methods, the CC-Net method views workflow modeling as a constraint satisfaction process. That is, describing the set of constraint recovery rules corresponding to the constraint violations is regarded as workflow modeling. The obtained set of workflow rules is free from process deadlock and considers all the events of triggering the workflow. We explore the use of the CC-Net method for the workflow modeling of a flexible manufacturing system.     

云制造环境下协同工作流控制模型研究

分析制造企业协同工作流管理的新需求,指出云制造模式下协同工作流管理存在自主、并发、协同、动态的特点。采用分层有限状态自动机对协同工作流控制过程建模,给出用于实现该模型的简化类结构设计,阐述模型中状态机的动态重构和行为控制过程。将该模型应用于某钢铁企业集团板材营销业务协同中,结果表明,该模型可提高业务协同效率及公司板材的接单能力。     

Abstract,link, publish,exploit: An end to end framework for workflow sharing

Scientific workflows are increasingly used to manage and share scientific computations and methods to analyze data. A variety of systems have been developed that store the workflows executed and make them part of public repositories However, workflows are published in the idiosyncratic format of the workflow system used for the creation and execution of the workflows. Browsing, linking and using the stored workflows and their results often becomes a challenge for scientists who may only be familiar with one system. In this paper we present an approach for addressing this issue by publishing and exploiting workflows as data on the Web with a representation that is independent from the workflow system used to create them. In order to achieve our goal, we follow the Linked Data Principles to publish workflow inputs, intermediate results, outputs and codes; and we reuse and extend well established standards like W3C PROV. We illustrate our approach by publishing workflows and consuming them with different tools designed to address common scenarios for workflow exploitation.     

支持自动调整的柔性工作流系统模型设计

目前的工作流管理系统仍然缺乏自动、及时的工作流调整功能。下一代的工作流管理系统应当提供足够的柔性来应对流程运行中出现的各种意外情况。提出的柔性工作流管理系统模型支持较为广泛的工作流自动调整。该工作流模型使用时间估计来决定哪些尚未执行的局部工作流受到意外情况的影响并且能够预先执行恰当的工作流调整。     

基于协作模式的工作流最优员工分配方法

一个业务流程的执行一般需要由多个员工共同协作完成.当员工完成流程中某项任务的能力已知时,员工之间的协作能力对于整个流程的执行性能就会有决定性的影响.通常,流程中执行活动的员工之间的协作能力越高,整个流程实例的运行效率就会越高.文中提出了一种基于协作模式的最优员工分配方法.该方法首先通过分析历史流程日志计算不同员工在执行不同活动时彼此之间的协作能力,然后从历史日志中挖掘出协作较好的员工分配方式（即,协作水平较高的协作模式）,再使用编码的方式将这些模式与待分配流程快速匹配选出可使流程协作水平达到最优的员工分配方式.实验说明该方法能够快速有效地实现流程协作最优的员工分配.     

基于Agent的协同设计中的工作流管理系统

针对CAD协同设计的特点，结合状态图和Agent技术的优势，提出了基于Agent的状态图工作流管理模式，讨论了其关键技术和实现方式，并以建筑设计为例进行了验证，为协同设计中的工作流管理提供了有益的参考。