BIGDATA: IA: Democratizing Massive Fluid Flow Simulations via Open Numerical Laboratories and Applications to Turbulent Flow and Geophysical Modeling

BIGDATA：IA：通过开放数值实验室以及湍流和地球物理建模应用使大规模流体流动模拟大众化

• 批准号: 1633124
• 负责人: Charles Meneveau
• 金额: $ 95.26万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2020-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1633124&HistoricalAwards=false
• 关键词: BIGDATA; IA; Democratizing; Massive; Fluid

Computer simulations of turbulent fluid flows are playing an increasingly vital role in engineering applications (e.g. reducing drag forces on vehicles and predicting wind turbine aerodynamic efficiency) and in geophysical sciences (e.g. describing the fate of pollutant dispersion or Lagrangian transport and mixing in the ocean). Simulations consist of discretizing and integrating the partial differential equations governing fluid flow and transport forward in time, providing solutions for physical variables (fields such as velocity and pressure) as function of time and space in the entire domain of interest. Since such simulations generate enormous amounts of data, the prevailing approach has been for researchers to analyze the data "on the fly" during the simulation runs while only a small subset of time-steps are stored for subsequent analysis. As a result, often large simulations of the same process must be repeated after new questions arise that were not initially obvious. Many (or even most) breakthrough concepts cannot be anticipated in advance, as they will be motivated in part by output data and must then be tested against it. As a result, there is a need for methods to store entire space-time data from such simulations. This project develops innovative tools for the efficient creation of open numerical databases that contain massive outputs from computational fluid dynamics simulations used in turbulence research and geophysical transport modeling and makes these available to the entire community. Several of the datasets to be included into the Open Numerical Laboratory will be contributed by external researchers. In addition to enhancing engineering and geophysical fluid mechanics and turbulence research, democratized access to large-scale turbulent flow simulation data will also play a crucial role in education and training for the next generation of researchers. Active learning through new educational modules that allow students to query simulation datasets in unprecedented detail will provide new educational paradigms. More broadly, the lessons learned from this project will be generalizable to many other fields where numerical simulations generate very large datasets that are difficult to access using prevailing approaches. In this way, the project will enhance the scientific and broader impacts of the US high-performance scientific computing infrastructure. This project will develop innovative tools for the efficient creation of open numerical databases that contain massive outputs from computational fluid dynamics simulations used in turbulence research and geophysical transport modeling. An ingest pipeline to be developed will enable users to transfer data from file systems containing the output of their massive direct numerical simulations, build a database, and serve it to the community for open exploratory data analysis and innovative turbulence and oceanic mixing research. To date, the investigators involved in this project have built an Open Numerical Laboratory focusing on direct numerical simulations (DNS) of canonical turbulent flows, in which the entire space-time data are available to the wider research community. However, the existing datasets are few in number and databases have been created one by one, using methodologies difficult to replicate on a massive scale. Moreover, emerging Exascale simulations will potentially result in data sets of unprecedented scale (tens to hundreds of PetaBytes). Advanced computer science algorithms will be required to tackle these challenges. This project will (a) develop automated, and scalable data management algorithms to ingest, index and serve very large data sets generated by a wide range of groups, (b) explore novel algorithms using spatio-temporal subsampling combined with online interpolation with re-simulation, yielding large compression factors depending on the subsampling stride, and (c) use machine learning algorithms to identify localized regions of interest in the simulations and save these 4D domains in a database for detailed follow-up analytics. The new databases will include data from (1) the largest channel flow DNS, (2) rotating and stratified turbulence of geophysical interest, (3) a DNS of developing wall boundary layer and (4) detailed ocean circulation models with complex boundary conditions. As part of the innovative domain science applications, data sets will be used to improve turbulence models using data-assimilation concepts, study Lagrangian vortex dynamics, and explore geophysical transport in a regional general circulation model of the North Atlantic Ocean.

湍流的计算机模拟在工程应用（例如减少车辆上的阻力和预测风力涡轮机的空气动力学效率）和地球物理科学（例如描述污染物扩散或海洋中的拉格朗日输运和混合的命运）中发挥着越来越重要的作用。模拟包括离散化和集成控制流体流动和运输的偏微分方程，在整个感兴趣的域中提供作为时间和空间函数的物理变量（如速度和压力）的解决方案。由于这种模拟会产生大量的数据，研究人员普遍采用的方法是在模拟运行期间“实时”分析数据，而只存储一小部分时间步长用于后续分析。因此，在出现最初并不明显的新问题后，必须重复对同一过程的大型模拟。许多（甚至大多数）突破性概念无法提前预测，因为它们将部分受到输出数据的激励，然后必须对其进行测试。因此，需要存储来自此类模拟的整个时空数据的方法。 该项目开发了创新工具，用于有效创建开放式数值数据库，其中包含湍流研究和地球物理传输建模中使用的计算流体动力学模拟的大量输出，并将其提供给整个社区。开放数值实验室将包括一些数据集，这些数据集将由外部研究人员提供。除了加强工程和地球物理流体力学和湍流研究外，大规模湍流模拟数据的民主化获取也将在下一代研究人员的教育和培训中发挥至关重要的作用。通过新的教育模块进行主动学习，允许学生以前所未有的细节查询模拟数据集，将提供新的教育范式。更广泛地说，从这个项目中吸取的经验教训将推广到许多其他领域，在这些领域，数值模拟产生了很大的数据集，使用现行方法很难获得。通过这种方式，该项目将增强美国高性能科学计算基础设施的科学和更广泛的影响。该项目将开发创新工具，用于有效创建开放式数值数据库，其中包含湍流研究和地球物理传输建模中使用的计算流体动力学模拟的大量输出。即将开发的摄取管道将使用户能够从文件系统传输数据，其中包含大量直接数值模拟的输出，建立数据库，并将其提供给社区，用于开放的探索性数据分析和创新的湍流和海洋混合研究。迄今为止，参与该项目的研究人员已经建立了一个开放式数值实验室，专注于正则湍流的直接数值模拟（DNS），其中整个时空数据可供更广泛的研究界使用。然而，现有的数据集数量很少，数据库是一个接一个地建立的，使用的方法难以大规模复制。 此外，新兴的Exascale模拟可能会产生前所未有规模的数据集（数十到数百Petascale）。需要先进的计算机科学算法来应对这些挑战。该项目将（a）开发自动化和可扩展的数据管理算法，以摄取、索引和服务于由广泛的群体生成的非常大的数据集，（B）探索使用时空二次采样结合在线插值和重新模拟的新算法，根据二次采样步幅产生大的压缩因子，以及（c）使用机器学习算法来识别模拟中的局部感兴趣区域，并将这些4D域保存在数据库中用于详细的后续分析。新数据库将包括以下数据：（1）最大的通道流DNS，（2）地球物理学感兴趣的旋转和分层湍流，（3）发展壁边界层的DNS和（4）具有复杂边界条件的详细海洋环流模型。作为创新领域科学应用的一部分，数据集将用于使用数据同化概念改进湍流模型，研究拉格朗日涡旋动力学，并探索北大西洋区域环流模型中的地球物理传输。

项目成果

From SkyServer to SciServer

从 SkyServer 到 SciServer

• DOI: 10.1177/0002716217745816
• 发表时间: 2017
• 期刊: The ANNALS of the American Academy of Political and Social Science
• 作者: Szalay, Alexander S.

Lagrangian Perspective on the Origins of Denmark Strait Overflow

丹麦海峡溢流起源的拉格朗日视角

• DOI: 10.1175/jpo-d-19-0210.1
• 发表时间: 2020
• 期刊: Journal of Physical Oceanography
• 影响因子: 3.5
• 作者: Saberi, Atousa;Haine, Thomas W.;Gelderloos, Renske;Femke de Jong, M.;Furey, Heather;Bower, Amy
• 通讯作者: Bower, Amy

Multiscale analysis of the invariants of the velocity gradient tensor in isotropic turbulence

• DOI: 10.1103/physrevfluids.3.044604
• 发表时间: 2018-04-11
• 期刊: PHYSICAL REVIEW FLUIDS
• 影响因子: 2.7
• 作者: Danish, Mohammad;Meneveau, Charles
• 通讯作者: Meneveau, Charles

Atlantic-Origin Overflow Water in the East Greenland Current

东格陵兰海流中的大西洋起源溢流水

• DOI: 10.1175/jpo-d-18-0216.1
• 发表时间: 2019
• 期刊: Journal of Physical Oceanography
• 影响因子: 3.5
• 作者: Håvik, Lisbeth;Almansi, Mattia;Våge, Kjetil;Haine, Thomas W.
• 通讯作者: Haine, Thomas W.

High-Frequency Variability in the Circulation and Hydrography of the Denmark Strait Overflow from a High-Resolution Numerical Model

• DOI: 10.1175/jpo-d-17-0129.1
• 发表时间: 2017-12-01
• 期刊: JOURNAL OF PHYSICAL OCEANOGRAPHY
• 影响因子: 3.5
• 作者: Almansi, Mattia;Haine, Thomas W. N.;Mastropole, Dana
• 通讯作者: Mastropole, Dana