A characterization of workflow management systems for extreme-scale applications |
| |
| Affiliation: | 1. Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA;2. British Geological Survey, Lyell Centre, Edinburgh, UK;3. School of Informatics, University of Edinburgh, Edinburgh, UK;4. Department of Informatics and Telecommunications, University of Athens, Athens, Greece;5. Lawrence Livermore National Laboratory, Livermore, CA, USA;6. School of Computer Science, University of Manchester, Manchester, UK;1. Information Sciences Institute and Department of Computer Science, University of Southern California, United States;2. Facultad de Informática, Universidad Politécnica de Madrid, Spain;1. Computation Institute, University of Chicago & Argonne National Laboratory, Chicago, IL, USA;2. RADICAL Laboratory, Rutgers University, New Brunswick, NJ, USA;3. AMPLab, University of California, Berkeley, CA, USA;1. Laboratoire de Recherche en Informatique, Université Paris-Sud, CNRS UMR 8623, Université Paris-Saclay, Orsay, France;2. Inria, VirtualPlants, Montpellier, France;3. Inria, Zenith, Montpellier, France;4. University Paris-Dauphine, PSL Research University, CNRS, UMR 7243, Centre Lamsade, 75016 Paris, France;5. IRISA, Rennes, France;6. INRA, UMR729, MISTEA, F-34060 Montpellier, France;7. Nantes Academic Hospital, CHU de Nantes, France;8. Centre de Biophysique Moléculaire (CNRS UPR4301, Orléans), France;9. IRD, DIADE, F-34394 Montpellier, France;10. EnginesOn / INRIA, Rennes, France;11. Institut Pasteur, Unité Bioinformatique Evolutive, Centre de Bioinformatique, Biostatistique et Biologie Intégrative (C3BI, USR 3756 IP CNRS), Paris, France;12. Institut Pasteur, Hub Bioinformatique et Biostatistique, Centre de Bioinformatique, Biostatistique et Biologie Intégrative (C3BI, USR 3756 IP CNRS), Paris, France;13. Institut Pasteur, Centre d’Informatique pour la Biologie, Direction des Systèmes d’Information, Paris, France;14. CIRAD, UMR AGAP, Montpellier, France;15. CNRS, UMS 3601; Institut Français de Bioinformatique, IFB-core, Avenue de la Terrasse, F-91190 Gif-sur-Yvette, France;1. PESC/COPPE - Federal University of Rio de Janeiro, Rio de Janeiro, Brazil;2. CEFET/RJ - Federal Center of Technological Education, Rio de Janeiro, Brazil;3. Institute of Computing - Fluminense Federal University, Niteroi, Brazil |
| |
| Abstract: | Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today’s computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems. |
| |
| Keywords: | Scientific workflows Workflow management systems Extreme-scale computing |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
