Collaborative Research: OAC Core: CEAPA: A Systematic Approach to Minimize Compression Error Propagation in HPC Applications

合作研究：OAC 核心：CEAPA：一种最小化 HPC 应用中压缩错误传播的系统方法

• 批准号: 2211539
• 负责人: Dingwen Tao
• 金额: $ 25万
• 依托单位: Washington State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-15 至 2022-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211539&HistoricalAwards=false
• 关键词: Collaborative; Research; OAC; Core; CEAPA

Today’s high-performance computing (HPC) applications produce vast volumes of data for post-analysis, presenting a major storage and I/O burden for HPC systems. To significantly reduce this burden, researchers have explored to use lossy compression techniques. While lossy compression can effectively reduce the size of data, it also introduces errors to the compressed data that often lead to incorrect computation results. As a result, scientists hesitate to use lossy compression in their scientific research. Thus, there is a critical need to develop an effective method to identify compression strategies which minimize error impact for a diversity of programs. This project aims to develop a systematic approach that helps scientists automatically select a lossy compression algorithm with the lowest error impact based their HPC programs and target compression ratios. It also integrates educational and outreach activities including student training and development of new curriculum on trustworthy data reduction and dependable HPC systems. Modeling compression error propagation in HPC programs is challenging because existing lossy compressors are developed with distinct principles that generate largely different compression errors on diverse HPC data. This project includes four key thrusts: (1) developing an accurate and efficient fault injection infrastructure that integrates with the fault models of commonly used lossy compression algorithms; (2) designing a fine-grained approach to characterize error propagation in HPC programs through program analysis and deposition based on the data dependencies and life cycle of compressed data; (3) developing a predictive model using machine learning techniques to select a compression strategy that minimizes the error impact on a given program and compression ratio; and (4) integrating the technique with domain-specific error impact metrics in real-world HPC applications and demonstrates the effectiveness of the technique by selecting compression strategies that give low error impact for the same ratios. Not only this project has an enormous positive impact on HPC cyberinfrastructure, but it also helps redefine the optimization of lossy compression techniques with emphasis on both efficiency and error impact.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

今天的高性能计算（HPC）应用程序产生大量的数据用于后期分析，这给HPC系统带来了很大的存储和I/O负担。为了显著减轻这一负担，研究人员已经探索使用有损压缩技术。有损压缩虽然可以有效地减小数据的大小，但也会给压缩后的数据带来误差，往往导致不正确的计算结果。因此，科学家们对在科学研究中使用有损压缩犹豫不决。因此，迫切需要开发一种有效的方法来确定压缩策略，使各种程序的错误影响最小化。该项目旨在开发一种系统的方法，帮助科学家根据HPC程序和目标压缩比自动选择具有最低误差影响的有损压缩算法。它还整合了教育和推广活动，包括学生培训和开发关于可信数据减少和可靠的高性能计算系统的新课程。对HPC程序中的压缩误差传播建模是具有挑战性的，因为现有的有损压缩器是根据不同的原理开发的，在不同的HPC数据上产生很大不同的压缩误差。本项目包括四个重点工作：(1)开发准确、高效的断层注入基础设施，该基础设施与常用的有损压缩算法的断层模型相结合；(2)基于压缩数据的数据依赖关系和生命周期，通过程序分析和沉积，设计了一种细粒度方法来表征HPC程序中的错误传播；(3)使用机器学习技术开发预测模型，以选择压缩策略，使对给定程序和压缩比的错误影响最小化；(4)将该技术与实际HPC应用中的特定领域误差影响指标相结合，并通过选择相同比率下误差影响较小的压缩策略来证明该技术的有效性。该项目不仅对高性能计算网络基础设施产生了巨大的积极影响，而且还有助于重新定义有损压缩技术的优化，强调效率和错误影响。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Design of a Quantization-Based DNN Delta Compression Framework for Model Snapshots and Federated Learning

• DOI: 10.1109/tpds.2022.3230840
• 发表时间: 2023-03
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Haoyu Jin;Donglei Wu;Shuyu Zhang;Xiangyu Zou;Sian Jin;Dingwen Tao;Qing Liao;Wen Xia