Collaborative Research: OAC Core: Topology-Aware Data Compression for Scientific Analysis and Visualization

今天的大规模模拟正在产生大量的数据，这些数据正在彻底改变科学思维和实践。例如，融合模拟可以在一次运行中产生200 PB的数据，而气候模拟可以每16秒产生260 TB的数据，分辨率为1平方公里。随着数据生成速率和可用I/O带宽之间的差距不断扩大，数据存储和移动正在成为现场和事后分析和可视化方面的极端规模科学模拟的重要瓶颈。这种差异需要数据压缩，即在现场压缩大规模模拟数据，并在现场和/或事后解压缩数据以进行分析和探索。另一方面，从大规模模拟中提取洞察力的关键步骤包括定义，提取和评估感兴趣的特征。拓扑数据分析提供了强大的工具来捕捉湍流燃烧，天文学，气候科学，计算物理和化学，生态学的科学数据的功能。 虽然有损压缩被用来解决大数据的挑战，但大多数现有的有损压缩器是不可知的，因此无法保留对科学发现至关重要的拓扑特征。该项目旨在研究和开发先进的有损压缩技术和软件，以保留数据中的拓扑特征，用于极端尺度的现场和事后分析和可视化。该项目的成功将通过实现科学数据的高效和有效压缩，以及对其他科学和工程学科的影响尺度，推动宇宙学，气候和融合应用的科学研究。此外，该项目的研究成果将纳入可视化和并行处理课程，通过研究和培训讲习班传播，并用于吸引代表性不足的学生，以扩大对计算的参与。该项目通过为标量场和矢量场数据开发一套拓扑感知的数据压缩算法来解决极端规模科学模拟中的数据压缩，分析和可视化需求。这种算法有效地减少了数据的大小，同时保留了拓扑概念定义的关键特征。这个项目将定义和执行拓扑感知的约束，先进的有损压缩算法。在今天的数据压缩范例中，这种能力还没有被系统地研究。该项目将影响特定领域，包括计算科学，数据分析，数据压缩和可视化，以及更广泛的科学界。该项目的研究产品将作为公开可用的软件提供，以显着推进当前和即将到来的exascale系统的研究网络基础设施。该项目将通过在广泛的平台上实现高效和有效的压缩，促进宇宙学、气候和聚变以外的多个科学学科的新发现。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Today's large-scale simulations are producing vast amounts of data that are revolutionizing scientific thinking and practices. For instance, a fusion simulation can produce 200 petabytes of data in a single run, while a climate simulation can generate 260 terabytes of data every 16 seconds with a 1 square kilometer resolution. As the disparity between data generation rates and available I/O bandwidths continues to grow, data storage and movement are becoming significant bottlenecks for extreme-scale scientific simulations in terms of in situ and post hoc analysis and visualization. The disparity necessitates data compression, which compresses large-scale simulations data in situ, and decompresses data in situ and/or post hoc for analysis and exploration. On the other hand, a critical step in extracting insight from large-scale simulations involves the definition, extraction, and evaluation of features of interest. Topological data analysis has provided powerful tools to capture features from scientific data in turbulent combustion, astronomy, climate science, computational physics and chemistry, and ecology. While lossy compression is leveraged to address the big data challenges, most existing lossy compressors are agnostic of and thus fail to preserve topological features that are essential to scientific discoveries. This project aims to research and develop advanced lossy compression techniques and software that preserve topological features in data for in situ and post hoc analysis and visualization at extreme scales. The success of this project will promote scientific research on driving applications in cosmology, climate, and fusion by enabling efficient and effective compression for scientific data, and the impact scales to other science and engineering disciplines. Furthermore, the research products of this project will be integrated into visualization and parallel processing curricula, disseminated via research and training workshops, and used to attract underrepresented students for broadening participation in computing. This project tackles the data compression, analysis, and visualization needs in extreme-scale scientific simulations by developing a suite of topology-aware data compression algorithms for scalar field and vector field data. Such algorithms effectively reduce the size of data while preserving critical features defined by topological notions. This project will define and enforce topology-aware constraints over advanced lossy compression algorithms. Such capabilities have not been studied systematically within today’s data compression paradigm. This project will impact specific fields, including computational science, data analysis, data compression, and visualization, and the broader scientific community. The research products of this project will be delivered as publicly available software to significantly advance the research cyberinfrastructure for current and upcoming exascale systems. This project will foster novel discoveries in multiple scientific disciplines beyond cosmology, climate, and fusion by enabling efficient and effective compression on a wide range of platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.