Collaborative Research: OAC Core: CEAPA: A Systematic Approach to Minimize Compression Error Propagation in HPC Applications

合作研究：OAC 核心：CEAPA：一种最小化 HPC 应用中压缩错误传播的系统方法

• 批准号: 2211539
• 负责人: Dingwen Tao
• 金额: $ 25万
• 依托单位: Washington State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-15 至 2022-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211539&HistoricalAwards=false
• 关键词: Collaborative; Research; OAC; Core; CEAPA

Today’s high-performance computing (HPC) applications produce vast volumes of data for post-analysis, presenting a major storage and I/O burden for HPC systems. To significantly reduce this burden, researchers have explored to use lossy compression techniques. While lossy compression can effectively reduce the size of data, it also introduces errors to the compressed data that often lead to incorrect computation results. As a result, scientists hesitate to use lossy compression in their scientific research. Thus, there is a critical need to develop an effective method to identify compression strategies which minimize error impact for a diversity of programs. This project aims to develop a systematic approach that helps scientists automatically select a lossy compression algorithm with the lowest error impact based their HPC programs and target compression ratios. It also integrates educational and outreach activities including student training and development of new curriculum on trustworthy data reduction and dependable HPC systems. Modeling compression error propagation in HPC programs is challenging because existing lossy compressors are developed with distinct principles that generate largely different compression errors on diverse HPC data. This project includes four key thrusts: (1) developing an accurate and efficient fault injection infrastructure that integrates with the fault models of commonly used lossy compression algorithms; (2) designing a fine-grained approach to characterize error propagation in HPC programs through program analysis and deposition based on the data dependencies and life cycle of compressed data; (3) developing a predictive model using machine learning techniques to select a compression strategy that minimizes the error impact on a given program and compression ratio; and (4) integrating the technique with domain-specific error impact metrics in real-world HPC applications and demonstrates the effectiveness of the technique by selecting compression strategies that give low error impact for the same ratios. Not only this project has an enormous positive impact on HPC cyberinfrastructure, but it also helps redefine the optimization of lossy compression techniques with emphasis on both efficiency and error impact.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当今的高性能计算（HPC）应用程序会产生大量的数据用于后期分析，这给HPC系统带来了巨大的存储和I/O负担。为了显著减少这种负担，研究人员已经探索使用有损压缩技术。虽然有损压缩可以有效地减少数据的大小，但它也会向压缩数据引入错误，这些错误通常会导致不正确的计算结果。因此，科学家们对在科学研究中使用有损压缩犹豫不决。因此，迫切需要开发一种有效的方法来识别压缩策略，使错误对各种程序的影响最小化。该项目旨在开发一种系统化的方法，帮助科学家根据HPC程序和目标压缩比自动选择具有最低错误影响的有损压缩算法。它还整合了教育和推广活动，包括学生培训和开发关于可靠数据简化和可靠HPC系统的新课程。在HPC程序中建模压缩误差传播是具有挑战性的，因为现有的有损压缩器是以不同的原理开发的，这些原理在不同的HPC数据上产生很大程度上不同的压缩误差。该项目主要包括四个方面的工作：（1）开发一个准确、高效的故障注入基础设施，该基础设施与常用有损压缩算法的故障模型相集成;（2）设计一种细粒度的方法，通过基于压缩数据的数据依赖性和生命周期的程序分析和沉积来表征HPC程序中的错误传播;（3）使用机器学习技术来开发预测模型，以选择最小化对给定程序和压缩比的错误影响的压缩策略;以及（4）在真实的中将该技术与特定于域的错误影响度量相结合。世界HPC应用程序，并证明了该技术的有效性，通过选择压缩策略，使低错误的影响相同的比率。该项目不仅对HPC网络基础设施产生了巨大的积极影响，而且还有助于重新定义有损压缩技术的优化，重点关注效率和错误影响。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Design of a Quantization-Based DNN Delta Compression Framework for Model Snapshots and Federated Learning

• DOI: 10.1109/tpds.2022.3230840
• 发表时间: 2023-03
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Haoyu Jin;Donglei Wu;Shuyu Zhang;Xiangyu Zou;Sian Jin;Dingwen Tao;Qing Liao;Wen Xia