Collaborative Research: Dynamic Staging Architecture for Accelerating I/O Pipelines

协作研究:加速 I/O 管道的动态分级架构

基本信息

  • 批准号:
    0937690
  • 负责人:
  • 金额:
    $ 13.39万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2010
  • 资助国家:
    美国
  • 起止时间:
    2010-04-01 至 2013-12-31
  • 项目状态:
    已结题

项目摘要

Petascale applications are producing terabytes of data at a great rate. Storage systems in large-scale machines are significantly stressed as I/O rates are not growing as fast to cope with data production. A variety of HPC activities such as writing output and checkpoint data are all stymied by the I/O bandwidth bottleneck. Further to this, the post-processing and subsequent analysis/visualization of computational results is increasingly time consuming due to the widening gap between the storage/processing capacities of supercomputers and users' local clusters. This research focuses on building a novel in-job dynamic data staging architecture and in bringing it to bear on the looming petascale I/O crisis. To this end, the following objectives are investigated: (i) the concerted use of node-local memory and emerging hardware such as Solid State Disks (SSDs), from a dedicated set of nodes, as a means to alleviate the I/O bandwidth bottleneck, (ii) the multiplexing of traditional user post-processing pipelines and secondary computations with asynchronous I/O on the staging ground to perform scalable I/O and data analytics, (iii) bypassing memory to access the staging area, and (iv) enabling QoS both in the staging ground and in the communication channel connecting it to compute client and persistent storage. This study will have a wide-ranging impact on future provisioning of extreme-scale machines and will provide formative guidelines to this end. The result of this research will be a set of integrated techniques that can fundamentally change the current parallel I/O model and accelerate petascale I/O pipelines. Further, this research will help analyze the utility of SSDs in day-to-day supercomputing I/O and inform the wider HPC community of its viability.
Petascale应用程序正在以很高的速度产生TB级的数据。大规模机器中的存储系统压力很大,因为I/O速率的增长速度不足以科普数据生产。各种HPC活动(如写入输出和检查点数据)都受到I/O带宽瓶颈的阻碍。此外,由于超级计算机和用户本地集群的存储/处理能力之间的差距越来越大,计算结果的后处理和后续分析/可视化越来越耗时。本研究的重点是建立一个新的工作动态数据分级架构,并使其承担迫在眉睫的千万亿I/O危机。为此,研究了以下目标:(i)协调使用来自专用节点集的节点本地存储器和新兴硬件(诸如固态硬盘(SSD)),作为缓解I/O带宽瓶颈的手段,(ii)传统的用户后处理流水线和二级计算与登台上的异步I/O的复用,以执行可扩展的I/O和数据分析,(iii)绕过存储器以访问暂存区域,以及(iv)在暂存地和将其连接到计算客户端和持久存储器的通信信道中启用QoS。这项研究将对未来提供极端规模的机器产生广泛的影响,并将为此提供形成性的指导方针。这项研究的结果将是一套集成的技术,可以从根本上改变目前的并行I/O模型和加速千万亿次I/O流水线。此外,这项研究将有助于分析SSD在日常超级计算I/O中的实用性,并向更广泛的HPC社区通报其可行性。

项目成果

期刊论文数量(0)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Xiaosong Ma其他文献

Adaptive Request Scheduling for Parallel Scientific Web Services
并行科学 Web 服务的自适应请求调度
  • DOI:
    10.1007/978-3-540-69497-7_19
  • 发表时间:
    2008
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Heshan Lin;Xiaosong Ma;Jiangtian Li;Ting Yu;N. Samatova
  • 通讯作者:
    N. Samatova
A new fast moisture sensitivity analysis method using equal moisture distribution simulation
一种基于等湿分布模拟的快速湿敏分析方法
Natural variation of Alfin‐like family affects seed size and drought tolerance in rice
Alfin类家族的自然变异影响水稻种子大小和耐旱性
  • DOI:
    10.1111/tpj.16003
  • 发表时间:
    2022
  • 期刊:
  • 影响因子:
    0
  • 作者:
    Yunan Yang;Xiaosong Ma;Hui Xia;Lei Wang;Shoujun Chen;Kai Xu;Fangwen Yang;Yuqiao Zou;Yulan Wang;Jinmin Zhu;Tianfei Li;Zhi Luo;Songping Hu;Zhigang Liao;Lijun Luo;Shunwu Yu
  • 通讯作者:
    Shunwu Yu
High-volume fly ash mortar mixes containing waste dolomite powders: Early-age properties, hydration characteristics, microstructure, and sustainability assessment
含废弃白云石粉的大掺量粉煤灰砂浆混合料:早期性能、水化特性、微观结构及可持续性评估
  • DOI:
    10.1016/j.conbuildmat.2025.141899
  • 发表时间:
    2025-07-18
  • 期刊:
  • 影响因子:
    8.000
  • 作者:
    Haibo Hu;Xiaosong Ma;Junqing Zuo;Anming She;Yongqi Wei;Wu Yao;Min Wu
  • 通讯作者:
    Min Wu
Overexpressing emOsNF-YB12/em elevated the content of jasmonic acid and impaired drought tolerance in rice
过表达emOsNF-YB12/em提高了水稻中茉莉酸的含量并削弱了水稻的耐旱性。
  • DOI:
    10.1016/j.plantsci.2025.112397
  • 发表时间:
    2025-03-01
  • 期刊:
  • 影响因子:
    4.100
  • 作者:
    Zhijuan Bian;Meiyao Chen;Lei Wang;Xiaosong Ma;Qing Yu;Zixin Jia;Jinghong Wu;Haibin Wei;Hongyan Liu;Yi Liu;Lijun Luo;Shunwu Yu
  • 通讯作者:
    Shunwu Yu

Xiaosong Ma的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Xiaosong Ma', 18)}}的其他基金

Collaborative Research: Automatic Extraction of Parallel I/O Benchmarks from HEC Applications
协作研究:从 HEC 应用程序中自动提取并行 I/O 基准
  • 批准号:
    0937908
  • 财政年份:
    2009
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Standard Grant
CAREER: Transparent, Interactive Desktop Parallel Computing for Scientific Data Processing
职业:用于科学数据处理的透明、交互式桌面并行计算
  • 批准号:
    0546301
  • 财政年份:
    2006
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: Application-adaptive I/O Stack for Data-intensive Scientific Computing
协作研究:用于数据密集型科学计算的应用自适应 I/O 堆栈
  • 批准号:
    0621470
  • 财政年份:
    2006
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Standard Grant
NGS: Collaborative Research: Cross-Platform Performance Estimation for Parallel Applications
NGS:协作研究:并行应用程序的跨平台性能评估
  • 批准号:
    0406305
  • 财政年份:
    2004
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Standard Grant

相似国自然基金

Research on Quantum Field Theory without a Lagrangian Description
  • 批准号:
    24ZR1403900
  • 批准年份:
    2024
  • 资助金额:
    0.0 万元
  • 项目类别:
    省市级项目
Cell Research
  • 批准号:
    31224802
  • 批准年份:
    2012
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Cell Research
  • 批准号:
    31024804
  • 批准年份:
    2010
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Cell Research (细胞研究)
  • 批准号:
    30824808
  • 批准年份:
    2008
  • 资助金额:
    24.0 万元
  • 项目类别:
    专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
  • 批准号:
    10774081
  • 批准年份:
    2007
  • 资助金额:
    45.0 万元
  • 项目类别:
    面上项目

相似海外基金

Collaborative Research: Chain Transform Fault: Understanding the dynamic behavior of a slow-slipping oceanic transform system
合作研究:链变换断层:了解慢滑海洋变换系统的动态行为
  • 批准号:
    2318855
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: CDS&E: data-enabled dynamic microstructural modeling of flowing complex fluids
合作研究:CDS
  • 批准号:
    2347345
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Standard Grant
Collaborative Research: Topological Defects and Dynamic Motion of Symmetry-breaking Tadpole Particles in Liquid Crystal Medium
合作研究:液晶介质中对称破缺蝌蚪粒子的拓扑缺陷与动态运动
  • 批准号:
    2344489
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Standard Grant
Collaborative Research: Dynamic connectivity of river networks as a framework for identifying controls on flux propagation and assessing landscape vulnerability to change
合作研究:河流网络的动态连通性作为识别通量传播控制和评估景观变化脆弱性的框架
  • 批准号:
    2342936
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: Dynamic connectivity of river networks as a framework for identifying controls on flux propagation and assessing landscape vulnerability to change
合作研究:河流网络的动态连通性作为识别通量传播控制和评估景观变化脆弱性的框架
  • 批准号:
    2342937
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: CDS&E: data-enabled dynamic microstructural modeling of flowing complex fluids
合作研究:CDS
  • 批准号:
    2347344
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Standard Grant
Collaborative Research: Chain Transform Fault: Understanding the dynamic behavior of a slow-slipping oceanic transform system
合作研究:链变换断层:了解慢滑海洋变换系统的动态行为
  • 批准号:
    2318851
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: AF: Medium: Fast Combinatorial Algorithms for (Dynamic) Matchings and Shortest Paths
合作研究:AF:中:(动态)匹配和最短路径的快速组合算法
  • 批准号:
    2402283
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: Chain Transform Fault: Understanding the dynamic behavior of a slow-slipping oceanic transform system
合作研究:链变换断层:了解慢滑海洋变换系统的动态行为
  • 批准号:
    2318854
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
Collaborative Research: AF: Medium: Fast Combinatorial Algorithms for (Dynamic) Matchings and Shortest Paths
合作研究:AF:中:(动态)匹配和最短路径的快速组合算法
  • 批准号:
    2402284
  • 财政年份:
    2024
  • 资助金额:
    $ 13.39万
  • 项目类别:
    Continuing Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了