Collaborative Research: Elements: ROCCI: Integrated Cyberinfrastructure for In Situ Lossy Compression Optimization Based on Post Hoc Analysis Requirements
合作研究:要素:ROCCI:基于事后分析要求的原位有损压缩优化的集成网络基础设施
基本信息
- 批准号:2247080
- 负责人:
- 金额:$ 28万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2022
- 资助国家:美国
- 起止时间:2022-10-01 至 2024-10-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
Today’s simulations and advanced instruments are producing vast volumes of data, presenting a major storage and I/O burden for scientists. Error-bounded lossy compressors, which can significantly reduce the data volume while controlling data distortion with a constant error bound, have been developed for years. However, a significant gap still remains in practice. On the one hand, the impact of the compression errors on scientific research is not well understood, so how to set an appropriate error bound for lossy compression is very challenging. On the other hand, how to select the best fit compression technology and run it automatically in scientific application codes is non-trivial because of strengths and weaknesses of different compression techniques and diverse characteristics of applications and datasets. This project aims to develop a Requirement-Oriented Compression Cyber-Infrastructure (ROCCI) for data-intensive domains such as astrophysics and materials science, which can select and run the best fit lossy compressor automatically at runtime, in terms of user's requirement on their post hoc analysis.The overarching goal of this project is to offer a complete series of automatic functions and services allowing users to transparently run the best fit compressor at runtime during the scientific simulations or data acquisition. This project advances knowledge and understanding with three key thrusts: (1) it builds an efficient layer to interoperate with different lossy compressors and diverse post hoc analysis requirements on data fidelity by leveraging an existing compression adaptor library (LibPressio) and compression assessment library (Z-checker); (2) it develops an efficient engine to determine the best fit compressor with optimized settings based on user’s post-hoc analysis requirements; and (3) it develops a user-friendly infrastructure that integrates compression optimization and execution via the HDF5 dynamic filter mechanism. This project particularly targets cosmology and materials science applications and their specific requirements of using lossy compressors in practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
今天的模拟和先进的仪器正在产生海量的数据,给科学家带来了巨大的存储和I/O负担。误差界有损压缩算法是近年来发展起来的一种新的有损压缩算法,它在控制数据失真的同时,显著地减少了数据量。然而,在实践中仍然存在着很大的差距。一方面,压缩误差对科学研究的影响还没有得到很好的了解,因此如何为有损压缩设置一个合适的误差界是非常具有挑战性的。另一方面,由于不同压缩技术的优缺点以及应用程序和数据集的不同特点,如何选择最合适的压缩技术并在科学应用程序代码中自动运行并不是一件容易的事情。该项目旨在为天体物理和材料科学等数据密集型领域开发一个面向需求的压缩数字基础设施(ROCCI),它可以根据用户对其后期分析的需求,在运行时自动选择和运行最佳匹配的有损压缩机。该项目的总体目标是提供一系列完整的自动化功能和服务,允许用户在科学模拟或数据采集过程中在运行时透明地运行最佳匹配的压缩机。这个项目通过三个关键的推力提高了知识和理解:(1)它通过利用现有的压缩适配器库(LibPressio)和压缩评估库(Z-Checker),建立了一个有效的层来与不同的有损压缩器和不同的特殊分析需求进行数据保真度的互操作;(2)它开发了一个高效的引擎,以根据用户的特殊分析需求来确定具有优化设置的最佳匹配的压缩机;以及(3)它开发了一个用户友好的基础设施,通过HDF5动态过滤机制集成了压缩优化和执行。该项目特别针对宇宙学和材料科学的应用及其在实践中使用有损压缩机的具体要求。该奖项反映了NSF的法定使命,并通过使用基金会的智力优势和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(8)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
cuZ-Checker: A GPU-Based Ultra-Fast Assessment System for Lossy Compressions
cuZ-Checker:基于 GPU 的有损压缩超快速评估系统
- DOI:10.1109/cluster48925.2021.00065
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Yu, Xiaodong;Di, Sheng;Gok, Ali Murat;Tao, Dingwen;Cappello, Franck
- 通讯作者:Cappello, Franck
Ultrafast Error-Bounded Lossy Compression for Scientific Datasets
科学数据集的超快误差限制有损压缩
- DOI:10.1145/3502181.3531473
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Yu, Xiaodong;Di, Sheng;Zhao, Kai;Tian, Jiannan;Tao, Dingwen;Liang, Xin;Cappello, Franck
- 通讯作者:Cappello, Franck
FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs
- DOI:10.1145/3588195.3592994
- 发表时间:2023-04
- 期刊:
- 影响因子:0
- 作者:Bo Zhang;Jiannan Tian;S. Di;Xiaodong Yu;Yunhe Feng;Xin Liang;Dingwen Tao;F. Cappello
- 通讯作者:Bo Zhang;Jiannan Tian;S. Di;Xiaodong Yu;Yunhe Feng;Xin Liang;Dingwen Tao;F. Cappello
Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs
优化 GPU 上科学数据的误差有限有损压缩
- DOI:10.1109/cluster48925.2021.00047
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Tian, Jiannan;Di, Sheng;Yu, Xiaodong;Rivera, Cody;Zhao, Kai;Jin, Sian;Feng, Yunhe;Liang, Xin;Tao, Dingwen;Cappello, Franck
- 通讯作者:Cappello, Franck
Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints
优化具有不同约束的科学数据的误差有限有损压缩
- DOI:10.1109/tpds.2022.3194695
- 发表时间:2022
- 期刊:
- 影响因子:5.3
- 作者:Liu, Yuanjian;Di, Sheng;Zhao, Kai;Jin, Sian;Wang, Cheng;Chard, Kyle;Tao, Dingwen;Foster, Ian;Cappello, Franck
- 通讯作者:Cappello, Franck
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Dingwen Tao其他文献
FastCLIP: A Suite of Optimization Techniques to Accelerate CLIP Training with Limited Resources
FastCLIP:一套优化技术,可利用有限的资源加速 CLIP 培训
- DOI:
- 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Xiyuan Wei;Fanjiang Ye;Ori Yonay;Xingyu Chen;Baixi Sun;Dingwen Tao;Tianbao Yang - 通讯作者:
Tianbao Yang
Z-checker: A framework for assessing lossy compression of scientific data
Z-checker:评估科学数据有损压缩的框架
- DOI:
- 发表时间:
2017 - 期刊:
- 影响因子:0
- 作者:
Dingwen Tao;S. Di;Hanqi Guo;Zizhong Chen;F. Cappello - 通讯作者:
F. Cappello
Extending checksum-based ABFT to tolerate soft errors online in iterative methods
扩展基于校验和的 ABFT 以容忍迭代方法中的在线软错误
- DOI:
- 发表时间:
2014 - 期刊:
- 影响因子:0
- 作者:
Longxiang Chen;Dingwen Tao;Panruo Wu;Zizhong Chen - 通讯作者:
Zizhong Chen
Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data
科学数据的相对误差有限有损压缩的性能优化
- DOI:
10.1109/tpds.2020.2972548 - 发表时间:
2020-07 - 期刊:
- 影响因子:5.3
- 作者:
Xiangyu Zou;Tao Lu;Wen Xia;Xuan Wang;Weizhe Zhang;Haijun Zhang;Sheng Di;Dingwen Tao;Franck Cappello - 通讯作者:
Franck Cappello
A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization
用于多分辨率科学数据简化和可视化的高质量工作流程
- DOI:
- 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Daoce Wang;Pascal Grosset;Jesus Pulido;Tushar M. Athawale;Jiannan Tian;Kai Zhao;Z. Lukic;Axel Huebl;Zhe Wang;James P. Ahrens;Dingwen Tao - 通讯作者:
Dingwen Tao
Dingwen Tao的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Dingwen Tao', 18)}}的其他基金
CAREER: A Highly Effective, Usable, Performant, Scalable Data Reduction Framework for HPC Systems and Applications
职业:适用于 HPC 系统和应用程序的高效、可用、高性能、可扩展的数据缩减框架
- 批准号:
2232120 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Frameworks: FZ: A fine-tunable cyberinfrastructure framework to streamline specialized lossy compression development
合作研究:框架:FZ:一个可微调的网络基础设施框架,用于简化专门的有损压缩开发
- 批准号:
2311876 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: SHF: Small: Reimagining Communication Bottlenecks in GNN Acceleration through Collaborative Locality Enhancement and Compression Co-Design
协作研究:SHF:小型:通过协作局部性增强和压缩协同设计重新想象 GNN 加速中的通信瓶颈
- 批准号:
2326495 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
CAREER: A Highly Effective, Usable, Performant, Scalable Data Reduction Framework for HPC Systems and Applications
职业:适用于 HPC 系统和应用程序的高效、可用、高性能、可扩展的数据缩减框架
- 批准号:
2312673 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
CDS&E: Collaborative Research: HyLoC: Objective-driven Adaptive Hybrid Lossy Compression Framework for Extreme-Scale Scientific Applications
CDS
- 批准号:
2303064 - 财政年份:2022
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
CRII: OAC: An Efficient Lossy Compression Framework for Reducing Memory Footprint for Extreme-Scale Deep Learning on GPU-Based HPC Systems
CRII:OAC:一种有效的有损压缩框架,可减少基于 GPU 的 HPC 系统上超大规模深度学习的内存占用
- 批准号:
2303820 - 财政年份:2022
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: OAC Core: CEAPA: A Systematic Approach to Minimize Compression Error Propagation in HPC Applications
合作研究:OAC 核心:CEAPA:一种最小化 HPC 应用中压缩错误传播的系统方法
- 批准号:
2211539 - 财政年份:2022
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: OAC Core: CEAPA: A Systematic Approach to Minimize Compression Error Propagation in HPC Applications
合作研究:OAC 核心:CEAPA:一种最小化 HPC 应用中压缩错误传播的系统方法
- 批准号:
2247060 - 财政年份:2022
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Elements: ROCCI: Integrated Cyberinfrastructure for In Situ Lossy Compression Optimization Based on Post Hoc Analysis Requirements
合作研究:要素:ROCCI:基于事后分析要求的原位有损压缩优化的集成网络基础设施
- 批准号:
2104024 - 财政年份:2021
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
CDS&E: Collaborative Research: HyLoC: Objective-driven Adaptive Hybrid Lossy Compression Framework for Extreme-Scale Scientific Applications
CDS
- 批准号:
2042084 - 财政年份:2020
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
相似国自然基金
Research on Quantum Field Theory without a Lagrangian Description
- 批准号:24ZR1403900
- 批准年份:2024
- 资助金额:0.0 万元
- 项目类别:省市级项目
Cell Research
- 批准号:31224802
- 批准年份:2012
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research
- 批准号:31024804
- 批准年份:2010
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Cell Research (细胞研究)
- 批准号:30824808
- 批准年份:2008
- 资助金额:24.0 万元
- 项目类别:专项基金项目
Research on the Rapid Growth Mechanism of KDP Crystal
- 批准号:10774081
- 批准年份:2007
- 资助金额:45.0 万元
- 项目类别:面上项目
相似海外基金
Collaborative Research: Elements: VLCC-States: Versioned Lineage-Driven Checkpointing of Composable States
协作研究:元素:VLCC-States:可组合状态的版本化谱系驱动检查点
- 批准号:
2411387 - 财政年份:2024
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Elements: Linking geochemical proxy records to crustal stratigraphic context via community-interactive cyberinfrastructure
合作研究:要素:通过社区交互式网络基础设施将地球化学代理记录与地壳地层背景联系起来
- 批准号:
2311092 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Elements: Lattice QCD software for nuclear physics on heterogeneous architectures
合作研究:Elements:用于异构架构核物理的 Lattice QCD 软件
- 批准号:
2311430 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Elements: ProDM: Developing A Unified Progressive Data Management Library for Exascale Computational Science
协作研究:要素:ProDM:为百亿亿次计算科学开发统一的渐进式数据管理库
- 批准号:
2311757 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: FuSe: Monolithic 3D Integration (M3D) of 2D Materials-Based CFET Logic Elements towards Advanced Microelectronics
合作研究:FuSe:面向先进微电子学的基于 2D 材料的 CFET 逻辑元件的单片 3D 集成 (M3D)
- 批准号:
2329189 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Experimental and computational constraints on the isotope fractionation of Mossbauer-inactive elements in mantle minerals
合作研究:地幔矿物中穆斯堡尔非活性元素同位素分馏的实验和计算约束
- 批准号:
2246686 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Experimental and computational constraints on the isotope fractionation of Mossbauer-inactive elements in mantle minerals
合作研究:地幔矿物中穆斯堡尔非活性元素同位素分馏的实验和计算约束
- 批准号:
2246687 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: GEO-CM: The occurrences of the rare earth elements in highly weathered sedimentary rocks, Georgia kaolins.
合作研究:GEO-CM:强风化沉积岩、乔治亚高岭土中稀土元素的出现。
- 批准号:
2327660 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: FuSe: Monolithic 3D Integration (M3D) of 2D Materials-Based CFET Logic Elements towards Advanced Microelectronics
合作研究:FuSe:面向先进微电子学的基于 2D 材料的 CFET 逻辑元件的单片 3D 集成 (M3D)
- 批准号:
2329192 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant
Collaborative Research: Elements: Linking geochemical proxy records to crustal stratigraphic context via community-interactive cyberinfrastructure
合作研究:要素:通过社区交互式网络基础设施将地球化学代理记录与地壳地层背景联系起来
- 批准号:
2311091 - 财政年份:2023
- 资助金额:
$ 28万 - 项目类别:
Standard Grant