Supporting complex workflows for data-intensive discovery collaboratively, reliably and efficiently

点击翻译按钮获取中文摘要

Scientific software capable of efficiently and effectively analyzing large-scale data has become necessary for research programs exploring some of the world's most pressing and complex challenges. As the volume, variety, and velocity of data increases, Big Data experiments making use of scientific workflow management systems (SWfMSs) to explore data, plan experimental execution, and visualize the results are becoming common and unavoidable. However, existing SWfMSs do not yet support the needs of the global scientific community to efficiently construct and adapt complex workflows, synchronously and asynchronously. As well, SWfMSs poorly handle group awareness and workflow provenance necessary for effective collaboration among distributed scientists, tracking of changes and debugging errors in workflows. The proposed research addresses these shortcomings by giving scientists methods to collaboratively and easily model their scientific experiments, understand and debug workflows, help them to recover from errors quickly, and increase their productivity. The long-term objective of this research is to develop a support framework that allows multi-disciplinary scientists to share complex workflows for data-intensive discoveries. I aim to provide a user friendly, reliable, collaborative and scalable computational environment for supporting large-scale scientific experiments. I focus on the following three short term objectives: Objective 1: Supporting workflow composition for scientists; Objective 2: Facilitating collaboration in complex workflow composition and Objective 3: Supporting on demand human-centric provenance queries in workflows. These objectives will be achieved in two different domains (plant phenotyping and genotyping and source code analysis) to increase generality of the research. I chose these domains as they involve large sets of computational modules, datasets and multi-disciplinary researchers and having accesses of these three things are important to make the proposed research usable to the broader scientific community. The proposed research program will train 8 HQP: 1 PhD, 5 Masters and 2 undergraduate students. My HQP will collaborate with multidisciplinary scientists and/or industrial partners, experiencing the challenges scientists face daily and learning how they could address those challenges, as well as improving their professional skills and expanding their professional network. The results of this will significantly increase the productivity of scientists in several domains (including Global Climate Change and Water Security) and will pave the way for extending the ideas to other scientific domains. Furthermore, the proposed solutions will allow software engineers to leverage both the principles and methodologies of SE and HCI to design reusable software architecture, plugins, and graphical user interfaces, targeting the seamless interaction between scientists and complex workflows in data intensive discoveries.