Scientific workflows: Past,present and future

This special issue and our editorial celebrate 10 years of progress with data-intensive or scientific workflows. There have been very substantial advances in the representation of workflows and in the engineering of workflow management systems (WMS). The creation and refinement stages are now well supported, with a significant improvement in usability. Improved abstraction supports cross-fertilisation between different workflow communities and consistent interpretation as WMS evolve. Through such re-engineering the WMS deliver much improved performance, significantly increased scale and sophisticated reliability mechanisms. Further improvement is anticipated from substantial advances in optimisation. We invited papers from those who have delivered these advances and selected 14 to represent today’s achievements and representative plans for future progress. This editorial introduces those contributions with an overview and categorisation of the papers. Furthermore, it elucidates responses from a survey of major workflow systems, which provides evidence of substantial progress and a structured index of related papers. We conclude with suggestions on areas where further research and development is needed and offer a vision of future research directions.     

Operational building blocks for business process modelling

The modelling of business processes is a rather complex, difficult and therefore time-consuming process. The efficiency of this process can be significantly enhanced by employing reusable building blocks, rather than to describe business processes from scratch every time a model is being created or modified. The paper describes a reference model which identifies a set of generic building blocks needed for enterprise modelling and identifies their relationships. Further specialisation of these generic building blocks into a profession and/or industry branch oriented set enables business people to model their processes with a minimum requirement on IT know-how. The CIMOSA concepts and language have been selected as the basis for the reference model. Since CIMOSA is one of the basic concepts recognised in the relevant standardisation work, the presented approach also obeys very closely the relevant European and international standards.     

ClowdFlows: Online workflows for distributed big data mining

The paper presents a platform for distributed computing, developed using the latest software technologies and computing paradigms to enable big data mining. The platform, called ClowdFlows, is implemented as a cloud-based web application with a graphical user interface which supports the construction and execution of data mining workflows, including web services used as workflow components. As a web application, the ClowdFlows platform poses no software requirements and can be used from any modern browser, including mobile devices. The constructed workflows can be declared either as private or public, which enables sharing the developed solutions, data and results on the web and in scientific publications. The server-side software of ClowdFlows can be multiplied and distributed to any number of computing nodes. From a developer’s perspective the platform is easy to extend and supports distributed development with packages. The paper focuses on big data processing in the batch and real-time processing mode. Big data analytics is provided through several algorithms, including novel ensemble techniques, implemented using the map-reduce paradigm and a special stream mining module for continuous parallel workflow execution. The batch mode and real-time processing mode are demonstrated with practical use cases. Performance analysis shows the benefit of using all available data for learning in distributed mode compared to using only subsets of data in non-distributed mode. The ability of ClowdFlows to handle big data sets and its nearly perfect linear speedup is demonstrated.     

Scientific workflows for computational reproducibility in the life sciences: Status,challenges and opportunities

With the development of new experimental technologies, biologists are faced with an avalanche of data to be computationally analyzed for scientific advancements and discoveries to emerge. Faced with the complexity of analysis pipelines, the large number of computational tools, and the enormous amount of data to manage, there is compelling evidence that many if not most scientific discoveries will not stand the test of time: increasing the reproducibility of computed results is of paramount importance.The objective we set out in this paper is to place scientific workflows in the context of reproducibility. To do so, we define several kinds of reproducibility that can be reached when scientific workflows are used to perform experiments. We characterize and define the criteria that need to be catered for by reproducibility-friendly scientific workflow systems, and use such criteria to place several representative and widely used workflow systems and companion tools within such a framework. We also discuss the remaining challenges posed by reproducible scientific workflows in the life sciences. Our study was guided by three use cases from the life science domain involving in silico experiments.     

The implementation and evaluation of a recovery system for workflows

Workflow systems are popular in daily business processing. Since vulnerability cannot be totally removed from a workflow management system, successful attacks always happen and may inject malicious tasks or incorrect data into the workflow system. Moreover, legitimate tasks referring to the incorrect data will further corrupt more data objects in the system. As a result, the integrity level of the system can be seriously compromised. This problem cannot be efficiently solved by existing defense mechanisms, such as access control, intrusion detection, and checkpoints. In this paper, we propose a practical solution for on-line attack recovery of workflows. The recovery system discovers all damages caused by the malicious tasks and automatically repairs the damages based on data and control dependencies between workflow tasks. We describe fundamental theories for workflow attack recovery system. Based on these theories, we build a prototype system and develop the corresponding recovery algorithms. We evaluate the performance of the recovery system under different attacking densities, intrusion detection delays and arrival rates. The experimental results show that our system is practical.     

Moduli-based language for building problem-solving scenarios

We propose a language for executive specification of problem-solving scenarios. The scenarios are an aggregate of actions realized as independent software moduli, the information connections between them and the sequence of their execution. An executive system controlling the execution of the scenarios was developed on the basis of the language. This system is the kernel of a system for automated control system design. The scenarios have a hierarchical structure. The organization of the scenarios is based on an object-oriented paradigm. Every module is an instance of a particular type module, called class. This allows one and the same module to be executed in different contexts. A unique information interaction mechanism between the moduli in the scenarios, allowing on-line changes in the information flow, was developed. The language is problem-independent, and can be used in various problem domains.     

Specification and validation of process constraints for flexible workflows

Workflow systems have traditionally focused on the so-called production processes which are characterized by pre-definition, high volume, and repetitiveness. Recently, the deployment of workflow systems in non-traditional domains such as collaborative applications, e-learning and cross-organizational process integration, have put forth new requirements for flexible and dynamic specification. However, this flexibility cannot be offered at the expense of control, a critical requirement of business processes.In this paper, we will present a foundation set of constraints for flexible workflow specification. These constraints are intended to provide an appropriate balance between flexibility and control. The constraint specification framework is based on the concept of “pockets of flexibility” which allows ad hoc changes and/or building of workflows for highly flexible processes. Basically, our approach is to provide the ability to execute on the basis of a partially specified model, where the full specification of the model is made at runtime, and may be unique to each instance.The verification of dynamically built models is essential. Where as ensuring that the model conforms to specified constraints does not pose great difficulty, ensuring that the constraint set itself does not carry conflicts and redundancy is an interesting and challenging problem. In this paper, we will provide a discussion on both the static and dynamic verification aspects. We will also briefly present Chameleon, a prototype workflow engine that implements these concepts.     

Desing and self-test for switched-current building blocks

This switched-current memory cell with a built-in self-test option serves as a building block for a range of analog functions. As an example application, the authors present a divide-by-two circuit for reference signal generation in algorithmic A/D converters. They also describe two self-test approaches for these building blocks and evaluate their effectiveness. The self-test functions are easy to apply, require very little overhead, and result in fault coverage up to 95% for shorts and 60% for open circuits. Analysis reveals that 100% testability may not be achievable in a cost-effective way for mixed-signal circuits     

A heterogeneous computing system for data mining workflows in multi-agent environments

Abstract: The computing-intensive data mining (DM) process calls for the support of a heterogeneous computing system, which consists of multiple computers with different configurations connected by a high-speed large-area network for increased computational power and resources. The DM process can be described as a multi-phase pipeline process, and in each phase there could be many optional methods. This makes the workflow for DM very complex and it can be modeled only by a directed acyclic graph (DAG). A heterogeneous computing system needs an effective and efficient scheduling framework, which orchestrates all the computing hardware to perform multiple competitive DM workflows. Motivated by the need for a practical solution of the scheduling problem for the DM workflow, this paper proposes a dynamic DAG scheduling algorithm according to the characteristics of an execution time estimation model for DM jobs. Based on an approximate estimation of job execution time, this algorithm first maps DM jobs to machines in a decentralized and diligent (defined in this paper) manner. Then the performance of this initial mapping can be improved through job migrations when necessary. The scheduling heuristic used considers the factors of both the minimal completion time criterion and the critical path in a DAG. We implement this system in an established multi-agent system environment, in which the reuse of existing DM algorithms is achieved by encapsulating them into agents. The system evaluation and its usage in oil well logging analysis are also discussed.     

Mining business process variants: Challenges, scenarios, algorithms

During the last years a new generation of process-aware information systems has emerged, which enables process model configurations at buildtime as well as process instance changes during runtime. Respective model adaptations result in a large number of model variants that are derived from the same process model, but slightly differ in structure. Generally, such model variants are expensive to configure and maintain. In this paper we address two scenarios for learning from process model adaptations and for discovering a reference model out of which the variants can be configured with minimum efforts. The first one is characterized by a reference process model and a collection of related process variants. The goal is to improve the original reference process model such that it fits better to the variant models. The second scenario comprises a collection of process variants, while the original reference model is unknown; i.e., the goal is to “merge” these variants into a new reference process model. We suggest two algorithms that are applicable in both scenarios, but have their pros and cons. We provide a systematic comparison of the two algorithms and further contrast them with conventional process mining techniques. Comparison results indicate good performance of our algorithms and also show that specific techniques are needed for learning from process configurations and adaptations. Finally, we provide results from a case study in automotive industry in which we successfully applied our algorithms.     

Scientific workflows in data analysis: Bridging expertise across multiple domains

In this paper, we demonstrate the use of scientific workflows in bridging expertise across multiple domains by re-purposing workflow fragments in the areas of text analysis, image analysis, and analysis of activity in video. We highlight how the reuse of workflows allows scientists to link across disciplines and avail themselves of the benefits of inter-disciplinary research beyond their normal area of expertise. In addition, we present in-depth studies of various tasks, including tasks for text analysis, multimedia analysis involving both images and text, video activity analysis, and analysis of artistic style using deep learning. These tasks show how the re-use of workflow fragments can turn a pre-existing, rudimentary approach into an expert-grade analysis. We also examine how workflow fragments save time and effort while amalgamating expertise in multiple areas such as machine learning and computer vision.     

Genetic circuit building blocks for cellular computation,communications, and signal processing

In this paper, we review an emerging engineering discipline to programcell behaviors by embedding synthetic gene networks that performcomputation, communications, and signal processing. To accomplishthis goal, we begin with a genetic component library and a biocircuitdesign methodology for assembling these components into compoundcircuits. The main challenge in biocircuit design lies in selectingwell-matched genetic components that when coupled, reliably producethe desired behavior. We use simulation tools to guide circuitdesign, a process that consists of selecting the appropriatecomponents and genetically modifying existing components until thedesired behavior is achieved. In addition to such rational design, wealso employ directed evolution to optimize genetic circuitbehavior. Building on Nature's fundamental principle of evolution,this unique process directs cells to mutate their own DNA until theyfind gene network configurations that exhibit the desired systemcharacteristics. The integration of all the above capabilities infuture synthetic gene networks will enable cells to performsophisticated digital and analog computation, both asindividual entities and as part of larger cell communities. Thisengineering discipline and its associated tools will advance thecapabilities of genetic engineering, and allow us to harness cells fora myriad of applications not previously achievable.     

A policy-based process mining framework: mining business policy texts for discovering process models

Many organizations use business policies to govern their business processes, often resulting in huge amounts of policy documents. As new regulations arise such as Sarbanes-Oxley, these business policies must be modified to ensure their correctness and consistency. Given the large amounts of business policies, manually analyzing policy documents to discover process information is very time-consuming and imposes excessive workload. In order to provide a solution to this information overload problem, we propose a novel approach named Policy-based Process Mining (PBPM) to automatically extracting process information from policy documents. Several text mining algorithms are applied to business policy texts in order to discover process-related policies and extract such process components as tasks, data items, and resources. Experiments are conducted to validate the extracted components and the results are found to be very promising. To the best of our knowledge, PBPM is the first approach that applies text mining towards discovering business process components from unstructured policy documents. The initial research results presented in this paper will require more research efforts to make PBPM a practical solution.     

Reactive and dynamic local search for max-clique: Engineering effective building blocks

This paper presents the results of an ongoing investigation about how different algorithmic building blocks contribute to solving the maximum clique problem. We consider greedy constructions, plateau searches, and more complex schemes based on dynamic penalties and/or prohibitions, in particular the recently proposed technique of dynamic local search and the previously proposed reactive local search (RLS). We design a variation of the original RLS algorithm where the role of long-term memory (LTM) is increased (RLS-LTM). In addition, we consider in detail the effect of the low-level implementation choices on the CPU time per iteration. We present experimental results on randomly generated graphs with different statistical properties, showing the crucial effects of the implementation, the robustness of different techniques, and their empirical scalability.     

选课系统的构建模块及技术

随着通讯技术和计算机网络技术的不断发展,教学管理网络化是一种必然趋势,本文结合我院实际,介绍学生选课系统的构建模块及所用技术.     

The performance assessment framework (PPAFR) for RPA implementation in a loan application process using process mining

When a company decides to automate its business processes by means of RPA (Robotic Process Automation), there are two fundamental questions that need to be answered. Firstly, what activities should the company automate and what characteristics make them suitable for RPA. The aim of the presented research is to design and demonstrate a data-driven performance framework assessing the impact of RPA implementation using process mining (PPAFR). Firstly, we comment on and summarise existing trends in process mining and RPA. Secondly, we describe research objectives and methods following the Design Science Research Methodology. Then, we identify critical factors for RPA implementation and design process stages of PPAFR. We demonstrate the design on real data from a loan application process. The demonstration consists of a process discovery using process mining methods, process analysis, and process simulation with assessment of RPA candidates. Based on the research results, a redesign of the process is proposed with emphasis on RPA implementation. Finally, we discuss the usefulness of PPAFR by helping companies to identify potentially suitable activities for RPA implementation and not overestimating potential gains. Obtained results show that within the loan application process, waiting times are the main causes of extended cases. If the waiting times are generated internally, it will be much easier for the company to address them. If the automation is focused mainly on processing times, the impact of automation on the overall performance of the process is insignificant or very low. Moreover, the research identified several characteristics which have to be considered when implementing RPA due to the impact on the overall performance of the process.

     

过程工业中数据挖掘技术的应用

针对过程工业的复杂性及其数据特点,讨论了数据挖掘技术工业应用的基本思想,提出了复杂过程工业应用数据挖掘技术的基本策略。     

A comparison of using Taverna and BPEL in building scientific workflows: the case of caGrid

When the emergence of ‘service‐oriented science,’ the need arises to orchestrate multiple services to facilitate scientific investigation—that is, to create ‘science workflows.’ We present here our findings in providing a workflow solution for the caGrid service‐based grid infrastructure. We choose BPEL and Taverna as candidates, and compare their usability in the lifecycle of a scientific workflow, including workflow composition, execution, and result analysis. Our experience shows that BPEL as an imperative language offers a comprehensive set of modeling primitives for workflows of all flavors; whereas Taverna offers a dataflow model and a more compact set of primitives that facilitates dataflow modeling and pipelined execution. We hope that this comparison study not only helps researchers to select a language or tool that meets their specific needs, but also offers some insight into how a workflow language and tool can fulfill the requirement of the scientific community. Copyright © 2009 John Wiley & Sons, Ltd.