Collaborative Research: OAC Core: Topology-Aware Data Compression for Scientific Analysis and Visualization

今天的大规模模拟正在产生大量数据，这些数据正在给科学思维和实践带来革命性的变化。例如，核聚变模拟可以在一次运行中产生200 PB的数据，而气候模拟可以以1平方公里的分辨率每16秒产生260 TB的数据。随着数据生成速率和可用I/O带宽之间的差距继续扩大，数据存储和移动正成为在现场和事后分析和可视化方面极端规模科学模拟的重大瓶颈。这种差异需要数据压缩，它在现场压缩大规模模拟数据，并在现场和/或后期解压缩数据以用于分析和探索。另一方面，从大规模模拟中提取洞察力的关键步骤包括定义、提取和评估感兴趣的特征。拓扑数据分析为从湍流燃烧、天文学、气候科学、计算物理化学和生态学等科学数据中获取特征提供了强大的工具。虽然利用有损压缩来解决大数据挑战，但大多数现有的有损压缩器是不可知的，因此无法保留对科学发现至关重要的拓扑特征。该项目旨在研究和开发先进的有损压缩技术和软件，以保留数据中的拓扑特征，以便在极大范围内进行现场和事后分析和可视化。该项目的成功将促进推动宇宙学、气候和核聚变应用的科学研究，方法是对科学数据进行高效和有效的压缩，并将影响范围扩大到其他科学和工程学科。此外，该项目的研究成果将纳入可视化和并行处理课程，通过研究和培训讲习班传播，并用于吸引代表性不足的学生更广泛地参与计算。该项目通过开发一套针对标量场和矢量场数据的拓扑感知数据压缩算法来满足极端规模科学模拟中的数据压缩、分析和可视化需求。这样的算法有效地减少了数据的大小，同时保留了由拓扑概念定义的关键特征。该项目将定义并强制执行高级有损压缩算法上的拓扑感知约束。在今天的数据压缩范例中，这种能力还没有得到系统的研究。该项目将影响特定领域，包括计算科学、数据分析、数据压缩和可视化，以及更广泛的科学界。该项目的研究产品将作为公开可用的软件提供，以显著推进现有和即将到来的艾级系统的研究网络基础设施。该项目将通过在广泛的平台上实现高效和有效的压缩，在宇宙学、气候学和核聚变之外的多个科学学科中促进新的发现。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Today's large-scale simulations are producing vast amounts of data that are revolutionizing scientific thinking and practices. For instance, a fusion simulation can produce 200 petabytes of data in a single run, while a climate simulation can generate 260 terabytes of data every 16 seconds with a 1 square kilometer resolution. As the disparity between data generation rates and available I/O bandwidths continues to grow, data storage and movement are becoming significant bottlenecks for extreme-scale scientific simulations in terms of in situ and post hoc analysis and visualization. The disparity necessitates data compression, which compresses large-scale simulations data in situ, and decompresses data in situ and/or post hoc for analysis and exploration. On the other hand, a critical step in extracting insight from large-scale simulations involves the definition, extraction, and evaluation of features of interest. Topological data analysis has provided powerful tools to capture features from scientific data in turbulent combustion, astronomy, climate science, computational physics and chemistry, and ecology. While lossy compression is leveraged to address the big data challenges, most existing lossy compressors are agnostic of and thus fail to preserve topological features that are essential to scientific discoveries. This project aims to research and develop advanced lossy compression techniques and software that preserve topological features in data for in situ and post hoc analysis and visualization at extreme scales. The success of this project will promote scientific research on driving applications in cosmology, climate, and fusion by enabling efficient and effective compression for scientific data, and the impact scales to other science and engineering disciplines. Furthermore, the research products of this project will be integrated into visualization and parallel processing curricula, disseminated via research and training workshops, and used to attract underrepresented students for broadening participation in computing. This project tackles the data compression, analysis, and visualization needs in extreme-scale scientific simulations by developing a suite of topology-aware data compression algorithms for scalar field and vector field data. Such algorithms effectively reduce the size of data while preserving critical features defined by topological notions. This project will define and enforce topology-aware constraints over advanced lossy compression algorithms. Such capabilities have not been studied systematically within today’s data compression paradigm. This project will impact specific fields, including computational science, data analysis, data compression, and visualization, and the broader scientific community. The research products of this project will be delivered as publicly available software to significantly advance the research cyberinfrastructure for current and upcoming exascale systems. This project will foster novel discoveries in multiple scientific disciplines beyond cosmology, climate, and fusion by enabling efficient and effective compression on a wide range of platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.