Supporting complex workflows for data-intensive discovery collaboratively, reliably and efficiently

能够高效有效地分析大规模数据的科学软件，已经成为探索世界上一些最紧迫和最复杂挑战的研究项目所必需的。随着数据量、种类和速度的增加，使用科学工作流管理系统(SWfMS)来探索数据、计划实验执行和可视化结果的大数据实验变得普遍且不可避免。然而，现有的SWfMS还不支持全球科学界高效地同步和异步地构建和适应复杂的工作流的需求。此外，SWfMS对于分布式科学家之间的有效协作、跟踪更改和调试工作流中的错误所需的团队意识和工作流起源处理能力也很差。拟议的研究通过为科学家提供协作和轻松地对他们的科学实验进行建模、了解和调试工作流程、帮助他们快速从错误中恢复并提高生产率的方法来解决这些缺点。这项研究的长期目标是开发一个支持框架，允许多学科科学家共享数据密集型发现的复杂工作流程。我的目标是为支持大规模科学实验提供一个用户友好、可靠、协作和可扩展的计算环境。我把重点放在以下三个短期目标上：目标1：支持科学家的工作流组成；目标2：促进复杂工作流组成中的协作；目标3：在工作流中支持按需以人类为中心的来源查询。这些目标将在两个不同的领域(植物表型和基因分型以及源代码分析)实现，以增加研究的一般性。我之所以选择这些领域，是因为它们涉及大量的计算模块、数据集和多学科研究人员，能够访问这三个领域对于使拟议的研究对更广泛的科学界有用是很重要的。拟议的研究计划将培养8名HQP：1名博士，5名硕士和2名本科生。我的HQP将与多学科科学家和/或行业合作伙伴合作，体验科学家每天面临的挑战，学习他们如何应对这些挑战，以及提高他们的专业技能和扩大他们的专业网络。这一成果将显著提高科学家在几个领域(包括全球气候变化和水安全)的生产力，并将为将这些想法扩展到其他科学领域铺平道路。此外，建议的解决方案将允许软件工程师利用SE和HCI的原则和方法来设计可重用的软件体系结构、插件和图形用户界面，目标是在数据密集型发现中实现科学家和复杂工作流程之间的无缝交互。

Scientific software capable of efficiently and effectively analyzing large-scale data has become necessary for research programs exploring some of the world's most pressing and complex challenges. As the volume, variety, and velocity of data increases, Big Data experiments making use of scientific workflow management systems (SWfMSs) to explore data, plan experimental execution, and visualize the results are becoming common and unavoidable. However, existing SWfMSs do not yet support the needs of the global scientific community to efficiently construct and adapt complex workflows, synchronously and asynchronously. As well, SWfMSs poorly handle group awareness and workflow provenance necessary for effective collaboration among distributed scientists, tracking of changes and debugging errors in workflows. The proposed research addresses these shortcomings by giving scientists methods to collaboratively and easily model their scientific experiments, understand and debug workflows, help them to recover from errors quickly, and increase their productivity. The long-term objective of this research is to develop a support framework that allows multi-disciplinary scientists to share complex workflows for data-intensive discoveries. I aim to provide a user friendly, reliable, collaborative and scalable computational environment for supporting large-scale scientific experiments. I focus on the following three short term objectives: Objective 1: Supporting workflow composition for scientists; Objective 2: Facilitating collaboration in complex workflow composition and Objective 3: Supporting on demand human-centric provenance queries in workflows. These objectives will be achieved in two different domains (plant phenotyping and genotyping and source code analysis) to increase generality of the research. I chose these domains as they involve large sets of computational modules, datasets and multi-disciplinary researchers and having accesses of these three things are important to make the proposed research usable to the broader scientific community. The proposed research program will train 8 HQP: 1 PhD, 5 Masters and 2 undergraduate students. My HQP will collaborate with multidisciplinary scientists and/or industrial partners, experiencing the challenges scientists face daily and learning how they could address those challenges, as well as improving their professional skills and expanding their professional network. The results of this will significantly increase the productivity of scientists in several domains (including Global Climate Change and Water Security) and will pave the way for extending the ideas to other scientific domains. Furthermore, the proposed solutions will allow software engineers to leverage both the principles and methodologies of SE and HCI to design reusable software architecture, plugins, and graphical user interfaces, targeting the seamless interaction between scientists and complex workflows in data intensive discoveries.