CAREER: Enabling Progressive Data Analytics for High Performance Computing: Algorithms and System Support

该奖项全部或部分由2021年美国救援计划法案（公法117-2）资助。从仿真输出中快速提取新知识对于全国各地高性能计算（HPC）设施的计算科学至关重要。然而，由于模拟产生的数据量与以产生数据的速率对数据进行后处理的能力之间的差距越来越大，这变得越来越具有挑战性。该项目旨在探索数据的简化表示，其总体目标是为HPC应用程序实现科学感知和高度适应性的数据分析。该项目将创建新的算法和软件系统，并有利于美国当前和未来的网络基础设施以及众多数据密集型科学应用，如核聚变，天体物理学，燃烧，地球科学等，从而加强美国在这一领域的竞争力和领导地位。项目目标的成功将大大缩短HPC中心从各种科学和工程学科的科学模拟中获得新知识的时间，并显着增强HPC研究和教育。该项目将通过参与代表性不足的群体和一系列综合研究和教育活动来为社会做出贡献。该项目将开发算法和系统支持，其核心是利用多级数据表示来实现HPC系统上的渐进式数据分析。所提出的工作从根本上不同于传统的有损数据压缩，因为它可以保证和执行科学约束，并根据应用程序的需求和系统状态提高精度。该项目整合了算法，系统和应用程序的研究和教育活动，考虑到大规模存储的应用需求和架构趋势，以推进科学数据管理领域。更具体地说，该项目将在以下几个领域做出贡献：1）基于约束的数据分解; 2）利用差错控制的多级表示来优化HPC存储系统的性能; 3）提供跨层解决方案来减轻容器化环境中的性能变化，通过概率方法选择要检索的级别数来实现多处理器和多应用程序的协调; 4）生产科学应用的集成和评估。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Rapidly extracting new knowledge from simulation output is critical to the computational sciences at high performance computing (HPC) facilities across the country. However, this has become increasingly challenging due to the growing disparity between the volume of data produced by simulations and the ability to post process the data at the rate it is produced. This project aims to explore reduced representations of data with the overarching goal of achieving science aware and highly adaptable data analytics for HPC applications. The project will create new algorithms and software systems, and benefit the current and future cyberinfrastructure in the U.S. as well as numerous data intensive scientific applications, such as nuclear fusion, astrophysics, combustion, earth science, and others, thus reinforcing the competitiveness and leadership of the United States in this area. Success in the project goals will greatly reduce the time to new knowledge from scientific simulations across various science and engineering disciplines at HPC centers and significantly enhance HPC research and education. The project will contribute to society through engaging underrepresented groups and a set of integrated research and education activities.The project will develop algorithms and system support centered on the idea of leveraging multilevel data representations to enable progressive data analytics on HPC systems. The proposed work fundamentally differs from conventional lossy data compression in that it can guarantee and enforce scientific constraints and augment accuracy based upon applications needs and system state. The project has integrated research and educational activities in algorithms, systems, and applications, taking into account application requirements and architecture trends in large-scale storage to advance the field of scientific data management. More specifically, the project will make contributions in several areas: 1) constraint-based data decomposition; 2) exploiting error-controlled multilevel representations for performance optimization on HPC storage systems; 3) providing a cross-layer solution to mitigate performance variation in containerized environments, with multiprocessor and multi-application coordination achieved through a probabilistic method for selecting the number of levels to retrieve; and 4) integration and evaluation on production science applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.