BIGDATA: IA: Democratizing Massive Fluid Flow Simulations via Open Numerical Laboratories and Applications to Turbulent Flow and Geophysical Modeling

点击翻译按钮获取中文摘要

Computer simulations of turbulent fluid flows are playing an increasingly vital role in engineering applications (e.g. reducing drag forces on vehicles and predicting wind turbine aerodynamic efficiency) and in geophysical sciences (e.g. describing the fate of pollutant dispersion or Lagrangian transport and mixing in the ocean). Simulations consist of discretizing and integrating the partial differential equations governing fluid flow and transport forward in time, providing solutions for physical variables (fields such as velocity and pressure) as function of time and space in the entire domain of interest. Since such simulations generate enormous amounts of data, the prevailing approach has been for researchers to analyze the data "on the fly" during the simulation runs while only a small subset of time-steps are stored for subsequent analysis. As a result, often large simulations of the same process must be repeated after new questions arise that were not initially obvious. Many (or even most) breakthrough concepts cannot be anticipated in advance, as they will be motivated in part by output data and must then be tested against it. As a result, there is a need for methods to store entire space-time data from such simulations. This project develops innovative tools for the efficient creation of open numerical databases that contain massive outputs from computational fluid dynamics simulations used in turbulence research and geophysical transport modeling and makes these available to the entire community. Several of the datasets to be included into the Open Numerical Laboratory will be contributed by external researchers. In addition to enhancing engineering and geophysical fluid mechanics and turbulence research, democratized access to large-scale turbulent flow simulation data will also play a crucial role in education and training for the next generation of researchers. Active learning through new educational modules that allow students to query simulation datasets in unprecedented detail will provide new educational paradigms. More broadly, the lessons learned from this project will be generalizable to many other fields where numerical simulations generate very large datasets that are difficult to access using prevailing approaches. In this way, the project will enhance the scientific and broader impacts of the US high-performance scientific computing infrastructure. This project will develop innovative tools for the efficient creation of open numerical databases that contain massive outputs from computational fluid dynamics simulations used in turbulence research and geophysical transport modeling. An ingest pipeline to be developed will enable users to transfer data from file systems containing the output of their massive direct numerical simulations, build a database, and serve it to the community for open exploratory data analysis and innovative turbulence and oceanic mixing research. To date, the investigators involved in this project have built an Open Numerical Laboratory focusing on direct numerical simulations (DNS) of canonical turbulent flows, in which the entire space-time data are available to the wider research community. However, the existing datasets are few in number and databases have been created one by one, using methodologies difficult to replicate on a massive scale. Moreover, emerging Exascale simulations will potentially result in data sets of unprecedented scale (tens to hundreds of PetaBytes). Advanced computer science algorithms will be required to tackle these challenges. This project will (a) develop automated, and scalable data management algorithms to ingest, index and serve very large data sets generated by a wide range of groups, (b) explore novel algorithms using spatio-temporal subsampling combined with online interpolation with re-simulation, yielding large compression factors depending on the subsampling stride, and (c) use machine learning algorithms to identify localized regions of interest in the simulations and save these 4D domains in a database for detailed follow-up analytics. The new databases will include data from (1) the largest channel flow DNS, (2) rotating and stratified turbulence of geophysical interest, (3) a DNS of developing wall boundary layer and (4) detailed ocean circulation models with complex boundary conditions. As part of the innovative domain science applications, data sets will be used to improve turbulence models using data-assimilation concepts, study Lagrangian vortex dynamics, and explore geophysical transport in a regional general circulation model of the North Atlantic Ocean.