CAREER: A Highly Effective, Usable, Performant, Scalable Data Reduction Framework for HPC Systems and Applications

这个CAREER项目研究和开发新的算法和软件，以提高高性能计算（HPC）系统和应用程序的数据减少的效率，可用性，性能和可扩展性。它有助于在宇宙学、气候学、地震学和机器学习等许多领域为HPC应用程序提供大数据管理的网络基础设施（CI）。研究结果将通过开放源码软件包和在主要会议和期刊上发表的出版物广泛传播。一个综合的教育和推广计划旨在促进CI劳动力的发展，包括整合概念和在课程中使用数据简化，为本科生和研究生提供研究培训，以及为来自大学和国家实验室的科学家和工程师专门设计的培训计划。该CAREER项目通过全面的分析建模，架构性能优化。具体的科学贡献包括：（1）建立轻量级模型，准确估计基于预测的压缩的预测和编码阶段不同技术的压缩比和质量，并优化压缩配置，在压缩质量约束下最大化压缩比;（2）开发了新的高效预测器和无损编码方法，用于在GPU上对科学数据进行有损压缩，并进行了深入的架构优化，以实现高吞吐量和高比率;（3）将优化压缩与并行I/O和MPI库深度集成，通过一系列优化，提高了数据移动性能和HPC应用的可扩展性。这一研究议程的成功使科学家和工程师能够很好地应对日益严峻的科学数据爆炸挑战。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This CAREER project researches and develops novel algorithms and software to improve the efficacy, usability, performance, and scalability of data reduction for high-performance computing (HPC) systems and applications. It contributes to the cyberinfrastructure (CI) of big data management for HPC applications in many domains such as cosmology, climatology, seismology, and machine learning. The research findings will be widely disseminated through open-source software packages and publications in premier conferences and journals. An integrated educational and outreach program is designed to foster CI workforce development, including integration of concepts and use of data reduction in curricula, research training for undergraduate and graduate students, and a specially designed training program for scientists and engineers from universities and national labs.This CAREER project simultaneously addresses these four critical issues in scientific data reduction through comprehensive analytical modeling and architectural performance optimization. Specific scientific contributions include: (1) it builds lightweight models to accurately estimate the compression ratio and quality of different techniques in the prediction and encoding stages of prediction-based compression, and optimizes the compression configurations to maximize the compression ratio under compression quality constraints; (2) it develops new efficient predictors and lossless encoding methods for lossy compression of scientific data on GPUs with deep architectural optimizations to achieve both high throughput and ratio; and (3) it deeply integrates the optimized compression with parallel I/O and MPI libraries with a series of optimizations to improve the performance of data movements and the scalability of HPC applications. The success of this research agenda enables scientists and engineers to well address the increasingly severe challenge of scientific data explosion.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.