Pegasus, a workflow management system for science automation
Pegasus, a workflow management system for science automation
复制标题
DOI：
10.1016/j.future.2014.10.008
复制
发表时间：
2015-05-01
影响因子：
7.5
通讯作者：
Wenger, Kent
中科院分区：
文献类型：
--
作者：
Deelman, Ewa;Vahi, Karan;Wenger, Kent
现代科学通常需要执行大规模、多阶段的仿真和数据分析管道，以实现对复杂系统的研究。这些管道中涉及的计算和数据量需要可扩展的工作流管理系统，这些系统能够可靠有效地协调和自动化分布式计算资源上的数据移动和任务执行：校园集群，国家网络基础设施以及商业和学术云。本文描述了Pegasus工作流管理系统的设计、开发和演化，该系统将抽象的工作流描述映射到分布式计算基础设施上。Pegasus已经被科学家们使用了12年多，应用领域广泛，包括天文学、地震学、生物信息学、物理学等。本文提供了飞马座系统的综合视图，显示其功能，随着时间的推移，在响应应用需求和科学计算平台的发展。本文介绍了Pegasus如何在各种计算基础设施上实现可靠的、可扩展的工作流执行。(C)2014爱思唯尔有限公司版权所有。
Modern science often requires the execution of large-scale, multi-stage simulation and data analysis pipelines to enable the study of complex systems. The amount of computation and data involved in these pipelines requires scalable workflow management systems that are able to reliably and efficiently coordinate and automate data movement and task execution on distributed computational resources: campus clusters, national cyberinfrastructures, and commercial and academic clouds. This paper describes the design, development and evolution of the Pegasus Workflow Management System, which maps abstract workflow descriptions onto distributed computing infrastructures. Pegasus has been used for more than twelve years by scientists in a wide variety of domains, including astronomy, seismology, bioinformatics, physics and others. This paper provides an integrated view of the Pegasus system, showing its capabilities that have been developed over time in response to application needs and to the evolution of the scientific computing platforms. The paper describes how Pegasus achieves reliable, scalable workflow execution across a wide variety of computing infrastructures. (C) 2014 Elsevier B.V. All rights reserved.