Frameworks: Advanced Cyberinfrastructure for Sustainable Community Usage of Big Data from Numerical Fluid Dynamics Simulations

在大多数基于计算机模拟的研究中，流行的方法是执行大型模拟，这些模拟产生的数据非常多，以至于在模拟运行期间只能计算一些数量，而最多只能存储和共享几个有代表性的快照以供后续分析。存储整个时间演化过程会产生访问和分布瓶颈，这是一个普遍存在的挑战。该项目解决了高性能科学计算前沿学科之一的挑战，并扩大了高性能科学计算的影响：流体湍流。湍流研究需要改进的工具来模拟大气和海洋科学中的许多自然过程，并开发工程应用。该项目建立了易于访问和可用的世界级湍流模拟数据库，有助于弥合顶级计算机模拟器与更广泛的湍流数据用户社区之间日益扩大的资源差距。该项目为湍流数据库开发并实施了一个先进的网络基础设施框架，使各种工程、大气和海洋流动的流体湍流研究成为可能。新颖的服务包括用户可编程的服务器计算，高效的批处理工具，易于使用的数据检查，也允许用户存储和查询特定流模式的位置。该系统扩展了基于笔记本的、人工智能增强的数据分析的适用性，可以访问具有更复杂数据对象的多样化结构化数据库联盟，并包括用于增加雷诺数的各种工程和地球物理新流的数据集。它包含与遗留系统向后兼容的元素，以支持和维护与现有活跃用户社区的兼容性。研究培训和指导的重点是物理概念与大规模模拟数据集的计算机和计算科学方面之间的相互作用。超过2.5 pb的数据将用于支持湍流研究的突破。该奖项由先进网络基础设施办公室颁发，由工程理事会下属的国家科学基金会化学、生物工程、环境和运输系统部门联合支持；物理海洋学项目、物理与动态气象学项目，以及地球科学理事会综合与合作教育与研究部。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In most computer simulation-based research, the prevailing approach has been to perform large simulations that generate so much data that only some quantities can be computed during the simulation runs while at most a few representative snapshots are stored and shared for subsequent analysis. Storing the entire time evolution creates access and distribution bottlenecks that have been a pervasive challenge. This project addresses the challenge and broadens the impact of high-performance scientific computing in one of the disciplines at the frontier of high-performance scientific computing: fluid turbulence. Improved tools for turbulence research are required to model many natural processes in atmospheric and ocean sciences and to develop engineering applications. The project builds easily accessible and useable databases from world-class turbulence simulations that help bridge the increasing resource gap between top computer simulators and the wider turbulence data user community. The project develops and implements an advanced cyberinfrastructure framework for turbulence databases that enables ground-breaking research on fluid turbulence in various engineering, atmospheric and ocean flows. Novel services include user-programmable server computation, efficient batch processing tools, easy-to-use inspections of the data that also allow users to store and query the locations of specific flow patterns. The system extends the applicability of notebook-based, AI-augmented data analyses accessing a diverse federation of structured databases with more complex data objects, and includes datasets for various new flows of engineering and geophysical interest at increasing Reynolds numbers. It contains backwards compatible elements with a legacy system, to support and maintain compatibility with an existing active community of users. Research training and mentoring focus on the interplay between physical concepts and computer and computational science aspects of massive simulation-based datasets. Over 2.5 Petabytes of data will become available to support breakthroughs in turbulence research. This award by the Office of Advanced Cyberinfrastructure is jointly supported by the National Science Foundation Division of Chemical, Bioengineering, Environmental and Transport Systems within the Directorate for Engineering; and by the Physical Oceanography Program, Physical and Dynamic Meteorology Program, and the Division of Integrative and Collaborative Education and Research within the Directorate for Geosciences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.