A characterization of workflow management systems for extreme-scale applications
A characterization of workflow management systems for extreme-scale applications
复制标题
DOI：
10.1016/j.future.2017.02.026
复制
发表时间：
2017-10-01
影响因子：
7.5
通讯作者：
Deelman, Ewa
中科院分区：
文献类型：
--
作者：
da Silva, Rafael Ferreira;Filgueira, Rosa;Deelman, Ewa
计算任务执行的自动化是提高科学生产力的核心。在过去的几年里，科学工作流程已经被建立为一个重要的抽象，它捕获了大型和复杂的科学应用程序的数据处理和计算。通过允许科学家建模和表达整个数据处理步骤及其依赖关系，工作流管理系统将科学家从应用程序的细节中解脱出来，并在计算基础设施上管理其执行。随着当今处理大量数据的计算和数据科学应用程序对资源的需求不断增加，新一代高性能计算（通常称为极端规模计算）的发展势在必行，这将给工作流应用程序和管理系统的设计带来多重挑战。本文提出了一种新的工作流管理系统的特征，该特征通常与极端规模计算应用相关。我们根据工作流执行模型、异构计算环境和数据访问方法对15种流行的工作流管理系统进行了分类。本文还调查了工作流的应用，并确定了未来在超大规模工作流和管理系统道路上的研究差距。(C) 2017 Elsevier B.V.版权所有
Automation of the execution of computational tasks is at the heart of improving scientific productivity. Over the last years, scientific workflows have been established as an important abstraction that captures data processing and computation of large and complex scientific applications. By allowing scientists to model and express entire data processing steps and their dependencies, workflow management systems relieve scientists from the details of an application and manage its execution on a computational infrastructure. As the resource requirements of today's computational and data science applications that process vast amounts of data keep increasing, there is a compelling case for a new generation of advances in high-performance computing, commonly termed as extreme-scale computing, which will bring forth multiple challenges for the design of workflow applications and management systems. This paper presents a novel characterization of workflow management systems using features commonly associated with extreme-scale computing applications. We classify 15 popular workflow management systems in terms of workflow execution models, heterogeneous computing environments, and data access methods. The paper also surveys workflow applications and identifies gaps for future research on the road to extreme-scale workflows and management systems. (C) 2017 Elsevier B.V. All rights reserved.