Collaborative Research: SHF: Small: Model-driven Design and Optimization of Dataflows for Scientific Applications

协作研究：SHF：小型：科学应用数据流的模型驱动设计和优化

• 批准号: 2331152
• 负责人: Michela Taufer
• 金额: $ 42.4万
• 依托单位: University of Tennessee Knoxville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2331152&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Model

The increasing capability of high-performance computing (HPC), cloud computing, and edge computing systems directly translates into the ability to generate more data and execute more extended analyses, thus expanding the range of natural phenomena that scientists can study using dataflows in scientific domains such as chemistry, materials sciences, molecular biology, and drug design. At the same time, the steady growth in the complexity of these dataflows also results in new challenges in the effective composition of single data tasks into scalable dataflow pipelines. This project addresses these critical challenges by developing solutions to optimize dataflow pipelines across heterogeneous resources. This project builds a broader community of HPC experts, who will have a far-reaching impact on the efficient development of dataflow pipelines supporting scientific applications. The team of researchers promotes increased participation of underrepresented students, particularly women, through mentoring students in Systers (the organization for women in Electrical Engineering and Computer Science at the University of Tennessee Knoxville). Furthermore, the researchers develop data analytics training tailored for early career professionals and share the material with the Midwest Research Computing and Data Consortium and the attendees at the bi-annual NSF/TCPP (Technical Community on Parallel Processing) workshops on parallel and distributed computing education (EduPar). This project has four main research components. First, the project defines a taxonomy of common dataflow motifs used in scientific domains, ranging from simple producer-consumer pairs to complex pipelines with multiple producers and consumers, by mapping these motifs to real scientific applications. Second, the project designs a middleware layer to handle dataflow pipelines executing on HPC, cloud, and edge resources. Third, the project develops a 2-step model for mitigating pipelines that result in data loss and inefficiencies associated with the slowdown in data production or consumption in dataflow pipelines. Finally, the project trains a broader community to utilize the taxonomy, middleware, and model to optimize real scientific applications by identifying potential bottlenecks and making necessary adjustments to maximize pipeline efficiency and accuracy, continuously monitoring and optimizing pipelines to ensure the highest quality scientific output possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高性能计算（HPC）、云计算和边缘计算系统的不断增强的能力直接转化为生成更多数据和执行更多扩展分析的能力，从而扩大了科学家可以在化学、材料科学、分子生物学和药物设计等科学领域使用数据流研究的自然现象的范围。与此同时，这些数据流复杂性的稳步增长也给将单个数据任务有效地组合成可扩展的数据流管道带来了新的挑战。该项目通过开发跨异构资源优化数据流管道的解决方案来解决这些关键挑战。该项目建立了一个更广泛的高性能计算专家社区，他们将对支持科学应用的数据流管道的高效开发产生深远的影响。研究小组通过指导systems（田纳西州诺克斯维尔大学电子工程和计算机科学专业的女性组织）的学生，促进了代表性不足的学生，特别是女性的更多参与。此外，研究人员开发了适合早期职业专业人员的数据分析培训，并与中西部研究计算和数据联盟以及两年一度的NSF/TCPP（并行处理技术社区）并行和分布式计算教育研讨会（EduPar）的与会者分享材料。这个项目有四个主要的研究部分。首先，通过将这些主题映射到真实的科学应用程序，该项目定义了科学领域中使用的常见数据流主题的分类，范围从简单的生产者-消费者对到具有多个生产者和消费者的复杂管道。其次，该项目设计了一个中间件层来处理在HPC、云和边缘资源上执行的数据流管道。第三，该项目开发了一个两步模型，用于减少由于数据流管道中数据生产或消耗放缓而导致的数据丢失和效率低下。最后，该项目培训更广泛的社区，通过识别潜在的瓶颈并进行必要的调整，以最大限度地提高管道的效率和准确性，不断监测和优化管道，以确保尽可能高质量的科学产出，从而利用分类法、中间件和模型来优化实际的科学应用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。