Supporting complex workflows for data-intensive discovery collaboratively, reliably and efficiently

支持复杂的工作流程，以协作、可靠和高效的方式进行数据密集型发现

• 批准号: RGPIN-2021-04233
• 负责人: Roy, Banani
• 金额: $ 1.75万
• 依托单位: University of Saskatchewan
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=742942
• 关键词: Supporting; complex; workflows; data; intensive

Scientific software capable of efficiently and effectively analyzing large-scale data has become necessary for research programs exploring some of the world's most pressing and complex challenges. As the volume, variety, and velocity of data increases, Big Data experiments making use of scientific workflow management systems (SWfMSs) to explore data, plan experimental execution, and visualize the results are becoming common and unavoidable. However, existing SWfMSs do not yet support the needs of the global scientific community to efficiently construct and adapt complex workflows, synchronously and asynchronously. As well, SWfMSs poorly handle group awareness and workflow provenance necessary for effective collaboration among distributed scientists, tracking of changes and debugging errors in workflows. The proposed research addresses these shortcomings by giving scientists methods to collaboratively and easily model their scientific experiments, understand and debug workflows, help them to recover from errors quickly, and increase their productivity. The long-term objective of this research is to develop a support framework that allows multi-disciplinary scientists to share complex workflows for data-intensive discoveries. I aim to provide a user friendly, reliable, collaborative and scalable computational environment for supporting large-scale scientific experiments. I focus on the following three short term objectives: Objective 1: Supporting workflow composition for scientists; Objective 2: Facilitating collaboration in complex workflow composition and Objective 3: Supporting on demand human-centric provenance queries in workflows. These objectives will be achieved in two different domains (plant phenotyping and genotyping and source code analysis) to increase generality of the research. I chose these domains as they involve large sets of computational modules, datasets and multi-disciplinary researchers and having accesses of these three things are important to make the proposed research usable to the broader scientific community. The proposed research program will train 8 HQP: 1 PhD, 5 Masters and 2 undergraduate students. My HQP will collaborate with multidisciplinary scientists and/or industrial partners, experiencing the challenges scientists face daily and learning how they could address those challenges, as well as improving their professional skills and expanding their professional network. The results of this will significantly increase the productivity of scientists in several domains (including Global Climate Change and Water Security) and will pave the way for extending the ideas to other scientific domains. Furthermore, the proposed solutions will allow software engineers to leverage both the principles and methodologies of SE and HCI to design reusable software architecture, plugins, and graphical user interfaces, targeting the seamless interaction between scientists and complex workflows in data intensive discoveries.

能够高效分析大规模数据的科学软件已经成为探索世界上一些最紧迫和最复杂挑战的研究项目的必要条件。随着数据量、种类和速度的增加，利用科学工作流程管理系统（SWfMS）来探索数据、计划实验执行并可视化结果的大数据实验变得越来越普遍且不可避免。然而，现有的SWfMS尚未支持全球科学界的需求，以有效地构建和适应复杂的工作流程，同步和异步。同样，SWfMS处理组意识和工作流起源的能力也很差，这是分布式科学家之间有效协作、跟踪工作流中的更改和调试错误所必需的。拟议的研究通过为科学家提供方法来解决这些缺点，以协作和轻松地建模他们的科学实验，理解和调试工作流程，帮助他们快速从错误中恢复，并提高他们的生产力。这项研究的长期目标是开发一个支持框架，允许多学科科学家共享数据密集型发现的复杂工作流程。我的目标是提供一个用户友好，可靠，协作和可扩展的计算环境，以支持大规模的科学实验。我专注于以下三个短期目标：目标1：支持科学家的工作流组成;目标2：促进复杂工作流组成的协作和目标3：支持按需以人为中心的出处查询工作流。这些目标将在两个不同的领域（植物表型和基因型和源代码分析）实现，以增加研究的通用性。我之所以选择这些领域，是因为它们涉及大量的计算模块、数据集和多学科研究人员，并且能够访问这三样东西对于使所提出的研究可用于更广泛的科学界非常重要。该研究项目将培养8名HQP：1名博士，5名硕士和2名本科生。我的HQP将与多学科科学家和/或行业合作伙伴合作，体验科学家每天面临的挑战，学习他们如何应对这些挑战，以及提高他们的专业技能和扩大他们的专业网络。 其结果将大大提高科学家在几个领域（包括全球气候变化和水安全）的生产力，并为将这些想法扩展到其他科学领域铺平道路。此外，拟议的解决方案将允许软件工程师利用SE和HCI的原理和方法来设计可重用的软件架构，插件和图形用户界面，目标是科学家和数据密集型发现中的复杂工作流程之间的无缝交互。