Collaborative Research: OAC Core: Topology-Aware Data Compression for Scientific Analysis and Visualization

今天的大规模模拟正在产生大量的数据，这些数据正在彻底改变科学思维和实践。例如，一次核聚变模拟可以在一次运行中产生200pb的数据，而一次气候模拟可以在1平方公里的分辨率下每16秒产生260tb的数据。随着数据生成速率和可用I/O带宽之间的差距不断扩大，数据存储和移动正在成为现场和事后分析和可视化方面的极端规模科学模拟的重要瓶颈。这种差异需要对数据进行压缩，即在现场压缩大规模模拟数据，并在现场和/或事后对数据进行解压缩，以便进行分析和勘探。另一方面，从大规模模拟中提取洞察力的关键步骤包括对感兴趣的特征的定义、提取和评估。拓扑数据分析为从湍流燃烧、天文学、气候科学、计算物理和化学以及生态学等领域的科学数据中捕捉特征提供了强大的工具。虽然有损压缩被用来解决大数据挑战，但大多数现有的有损压缩器都无法识别，因此无法保留对科学发现至关重要的拓扑特征。该项目旨在研究和开发先进的有损压缩技术和软件，以保留数据的拓扑特征，以便在极端尺度下进行原位和事后分析和可视化。该项目的成功将促进科学研究，通过对科学数据进行高效和有效的压缩，推动在宇宙学、气候和核聚变等领域的应用，并对其他科学和工程学科产生影响。此外，该项目的研究成果将纳入可视化和并行处理课程，通过研究和培训讲习班传播，并用于吸引代表性不足的学生，以扩大对计算的参与。该项目通过为标量场和矢量场数据开发一套拓扑感知数据压缩算法，解决了极端尺度科学模拟中的数据压缩、分析和可视化需求。这种算法有效地减少了数据的大小，同时保留了由拓扑概念定义的关键特征。该项目将在高级有损压缩算法上定义和执行拓扑感知约束。在今天的数据压缩范例中，还没有对这种能力进行系统的研究。该项目将影响特定领域，包括计算科学、数据分析、数据压缩和可视化，以及更广泛的科学界。该项目的研究成果将作为公开可用的软件交付，以显著推进当前和即将推出的百亿亿级系统的研究网络基础设施。该项目将通过在广泛的平台上实现高效和有效的压缩，促进在宇宙学、气候和核聚变以外的多个科学学科的新发现。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Today's large-scale simulations are producing vast amounts of data that are revolutionizing scientific thinking and practices. For instance, a fusion simulation can produce 200 petabytes of data in a single run, while a climate simulation can generate 260 terabytes of data every 16 seconds with a 1 square kilometer resolution. As the disparity between data generation rates and available I/O bandwidths continues to grow, data storage and movement are becoming significant bottlenecks for extreme-scale scientific simulations in terms of in situ and post hoc analysis and visualization. The disparity necessitates data compression, which compresses large-scale simulations data in situ, and decompresses data in situ and/or post hoc for analysis and exploration. On the other hand, a critical step in extracting insight from large-scale simulations involves the definition, extraction, and evaluation of features of interest. Topological data analysis has provided powerful tools to capture features from scientific data in turbulent combustion, astronomy, climate science, computational physics and chemistry, and ecology. While lossy compression is leveraged to address the big data challenges, most existing lossy compressors are agnostic of and thus fail to preserve topological features that are essential to scientific discoveries. This project aims to research and develop advanced lossy compression techniques and software that preserve topological features in data for in situ and post hoc analysis and visualization at extreme scales. The success of this project will promote scientific research on driving applications in cosmology, climate, and fusion by enabling efficient and effective compression for scientific data, and the impact scales to other science and engineering disciplines. Furthermore, the research products of this project will be integrated into visualization and parallel processing curricula, disseminated via research and training workshops, and used to attract underrepresented students for broadening participation in computing. This project tackles the data compression, analysis, and visualization needs in extreme-scale scientific simulations by developing a suite of topology-aware data compression algorithms for scalar field and vector field data. Such algorithms effectively reduce the size of data while preserving critical features defined by topological notions. This project will define and enforce topology-aware constraints over advanced lossy compression algorithms. Such capabilities have not been studied systematically within today’s data compression paradigm. This project will impact specific fields, including computational science, data analysis, data compression, and visualization, and the broader scientific community. The research products of this project will be delivered as publicly available software to significantly advance the research cyberinfrastructure for current and upcoming exascale systems. This project will foster novel discoveries in multiple scientific disciplines beyond cosmology, climate, and fusion by enabling efficient and effective compression on a wide range of platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.