Collaborative Research: SHF: Small: Scalable and Extensible I/O Runtime and Tools for Next Generation Adaptive Data Layouts

协作研究:SHF:小型:可扩展和可扩展的 I/O 运行时以及下一代自适应数据布局的工具

基本信息

  • 批准号:
    2221812
  • 负责人:
  • 金额:
    $ 29.97万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2022
  • 资助国家:
    美国
  • 起止时间:
    2022-07-15 至 2025-06-30
  • 项目状态:
    未结题

项目摘要

The ability to perform large scale scientific simulations on supercomputers have fueled a wave of innovation and discoveries across a range of disciplines including energy, cosmology, earth science, medicine, and national security. With the advent of exascale, applications promise to deliver data of ever-increasing size at higher resolution and fidelity. Current technology trends in High Performance Computing (HPC) systems are creating an unprecedented gap between compute and I/O performance, making data movement the slowest component of the simulation-analysis pipeline. Many techniques have been proposed to alleviate this bottleneck including compression and hierarchical data layouts, but current solutions lack scalability and portability, and do not provide a holistic approach to the data-management needs of both parallel I/O and analysis (in situ and post-hoc) workflows. This work will develop a scalable and extensible I/O runtime and tools for the next-generation adaptive data layouts that inherently imbibe compression and progressive data access, advancing the state of art in the field of high-performance data management. The work will lay the foundation for an end-to-end data management solution that will cater to the challenging needs of the entire simulation-analysis pipeline and significantly accelerate science at exascale.The research aims to develop an end-to-end data-management solution for the next generation adaptive data layouts. The proposed data layouts will be hierarchical, compressed, and tunable, making them suitable to deal with the data deluge and the evolving landscape of HPC. A hierarchical layout will allow progressive access to massively large data enabling post-hoc and in situ analysis at any scale. State-of-the-art data compression and reduction techniques will significantly alleviate data-movement bottlenecks, especially while performing parallel I/O. Finally, a tunable layout combined with novel performance analysis and visualization tools will allow data-driven approaches to optimize I/O performance at runtime for different workflows and HPC platforms. This project aims to achieve its goals by developing: a scalable and tunable parallel I/O runtime that will support progressive read/write operations using adaptive data layouts; interfaces to support the adaptive data layouts for in situ workflows; a novel WebGPU-powered visualization system that can take advantage of the progressive nature of the layout enabling interactive exploration of large datasets on web browsers; and performance-mining and -visualization tools to enable data-driven and portable I/O performance prediction and auto-tuning. The solution will be evaluated on leadership supercomputers and mid-scale clusters, and integrated with large-scale simulations, analysis, and I/O frameworks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
在超级计算机上进行大规模科学模拟的能力推动了一系列学科的创新和发现,包括能源,宇宙学,地球科学,医学和国家安全。随着兆兆级的出现,应用程序承诺以更高的分辨率和保真度提供不断增加的数据。高性能计算(HPC)系统的当前技术趋势正在计算和I/O性能之间产生前所未有的差距,使数据移动成为模拟分析管道中最慢的组件。已经提出了许多技术来缓解这一瓶颈,包括压缩和分层数据布局,但目前的解决方案缺乏可扩展性和可移植性,并没有提供一个整体的方法来满足并行I/O和分析(原位和事后)工作流的数据管理需求。这项工作将为下一代自适应数据布局开发一个可扩展和可扩展的I/O运行时和工具,这些自适应数据布局本质上吸收了压缩和渐进式数据访问,推进了高性能数据管理领域的最新技术。这项工作将为端到端的数据管理解决方案奠定基础,该解决方案将满足整个模拟分析管道的挑战性需求,并显着加快科学在exascale。该研究旨在为下一代自适应数据布局开发端到端的数据管理解决方案。拟议的数据布局将是分层的,压缩的和可调的,使它们适合处理数据泛滥和HPC不断发展的景观。分层布局将允许逐步访问海量数据,从而能够进行任何规模的事后和现场分析。最先进的数据压缩和精简技术将显著缓解数据移动瓶颈,特别是在执行并行I/O时。最后,可调布局与新颖的性能分析和可视化工具相结合,将允许数据驱动的方法在运行时优化不同工作流和HPC平台的I/O性能。该项目旨在通过开发以下各项来实现其目标:一个可伸缩和可调的并行I/O运行时,将支持使用自适应数据布局的渐进式读/写操作;支持现场工作流程的自适应数据布局的接口;一个新颖的WebGPU驱动的可视化系统,可以利用布局的渐进性,在Web浏览器上交互式探索大型数据集;以及性能挖掘和可视化工具,以实现数据驱动和便携式I/O性能预测和自动调优。该解决方案将在领先的超级计算机和中型集群上进行评估,并与大规模模拟,分析和I/O框架集成。该奖项反映了NSF的法定使命,并通过使用基金会的智力价值和更广泛的影响审查标准进行评估,被认为值得支持。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Steve Petruzza其他文献

Configurable Algorithms for All-to-All Collectives
适用于所有集体的可配置算法
Blueprint: Cyberinfrastructure Center of Excellence
蓝图:网络基础设施卓越中心
  • DOI:
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    0
  • 作者:
    E. Deelman;A. Mandal;Angela P. Murillo;J. Nabrzyski;Valerio Pascucci;R. Ricci;I. Baldin;S. Sons;L. Christopherson;Charles Vardeman;Rafael Ferreira da Silva;J. Wyngaard;Steve Petruzza;M. Rynge;K. Vahi;W. Whitcup;Josh Drake;Erik Scott University of Southern California;University of North Carolina at Chapel Hill;Indiana University;U. Utah;U. N. Dame
  • 通讯作者:
    U. N. Dame
Lessons learned towards the immediate delivery of massive aerial imagery to farmers and crop consultants
立即向农民和作物顾问提供大量航空图像的经验教训
  • DOI:
    10.1117/12.2587694
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    3.9
  • 作者:
    A. Gooch;Steve Petruzza;A. Gyulassy;G. Scorzelli;Valerio Pascucci;L. Rantham;Weston Adcock;C. Coopmans
  • 通讯作者:
    C. Coopmans
Scientific timely actionable robotic data operating system (STARDOS): architecture and progress
科学及时可操作的机器人数据操作系统(STARDOS):架构和进展
  • DOI:
  • 发表时间:
    2021
  • 期刊:
  • 影响因子:
    0
  • 作者:
    C. Coopmans;Stephen Brimhall;R. Goodman;Steve Petruzza
  • 通讯作者:
    Steve Petruzza
A Task-Based Abstraction Layer for User Productivity and Performance Portability in Post-Moore ’ s Era Supercomputing
后摩尔时代超级计算中用户生产力和性能可移植性的基于任务的抽象层
  • DOI:
  • 发表时间:
    2018
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Steve Petruzza;A. Gyulassy;Valerio Pascucci;P. Bremer
  • 通讯作者:
    P. Bremer

Steve Petruzza的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

相似国自然基金

Research on Quantum Field Theory without a Lagrangian Description
  • 批准号:
    24ZR1403900
  • 批准年份:
    2024
  • 资助金额:
    0.0 万元
  • 项目类别:
    省市级项目
Cell Research
  • 批准号:
    31224802
  • 批准年份:
    2012
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Cell Research
  • 批准号:
    31024804
  • 批准年份:
    2010
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Cell Research (细胞研究)
  • 批准号:
    30824808
  • 批准年份:
    2008
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
  • 批准号:
    10774081
  • 批准年份:
    2007
  • 资助金额:
    45.0 万元
  • 项目类别:
    面上项目

相似海外基金

Collaborative Research: SHF: Medium: Differentiable Hardware Synthesis
合作研究:SHF:媒介:可微分硬件合成
  • 批准号:
    2403134
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Small: LEGAS: Learning Evolving Graphs At Scale
协作研究:SHF:小型:LEGAS:大规模学习演化图
  • 批准号:
    2331302
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Small: LEGAS: Learning Evolving Graphs At Scale
协作研究:SHF:小型:LEGAS:大规模学习演化图
  • 批准号:
    2331301
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Small: Efficient and Scalable Privacy-Preserving Neural Network Inference based on Ciphertext-Ciphertext Fully Homomorphic Encryption
合作研究:SHF:小型:基于密文-密文全同态加密的高效、可扩展的隐私保护神经网络推理
  • 批准号:
    2412357
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Medium: Enabling Graphics Processing Unit Performance Simulation for Large-Scale Workloads with Lightweight Simulation Methods
合作研究:SHF:中:通过轻量级仿真方法实现大规模工作负载的图形处理单元性能仿真
  • 批准号:
    2402804
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Medium: Tiny Chiplets for Big AI: A Reconfigurable-On-Package System
合作研究:SHF:中:用于大人工智能的微型芯片:可重新配置的封装系统
  • 批准号:
    2403408
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Medium: Toward Understandability and Interpretability for Neural Language Models of Source Code
合作研究:SHF:媒介:实现源代码神经语言模型的可理解性和可解释性
  • 批准号:
    2423813
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Medium: Enabling GPU Performance Simulation for Large-Scale Workloads with Lightweight Simulation Methods
合作研究:SHF:中:通过轻量级仿真方法实现大规模工作负载的 GPU 性能仿真
  • 批准号:
    2402806
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Medium: Differentiable Hardware Synthesis
合作研究:SHF:媒介:可微分硬件合成
  • 批准号:
    2403135
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
Collaborative Research: SHF: Medium: Tiny Chiplets for Big AI: A Reconfigurable-On-Package System
合作研究:SHF:中:用于大人工智能的微型芯片:可重新配置的封装系统
  • 批准号:
    2403409
  • 财政年份:
    2024
  • 资助金额:
    $ 29.97万
  • 项目类别:
    Standard Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了