ITR - ASE - (int+sim): Exploring the Lagrangian Structure of Complex Flows with 100 Terabyte Datasets

AST-0428325 Vishniac尖端科学已经变得非常数据密集型，因此无法存储和分析今天的数TB数据集（以及明天的数PB数据集）越来越限制科学进步。 一个重要的，长期存在的问题，被费曼描述为经典物理学中最后一个未解决的问题，是磁化和非磁化流体中的湍流。 一个难以克服的障碍是，湍流级联的内在物理学是在拉格朗日坐标系中发生的，因此，必须在时间上追溯流体粒子。 这一要求也适用于宇宙起源等其他领域，在这些领域中，星系需要通过过去的遭遇来追溯。本研究项目将开发一种新的大规模数值模拟方法，将几乎每一个时间步都保存在数据库中，从而允许通过相对简单的计算进行随后的分析，只需要大量的数据。 这个新想法已经过测试，它避免了将数据移动到计算节点的瓶颈，而是将低级的索引数据访问与数据库内部发生的处理相结合。 这种方法允许对湍流问题进行预期的新颖攻击，将前沿科学项目与未来模拟环境的演示相结合。 它要求开发工具和程序，以多种格式组织数据，每种格式都适合于特定类型的统计分析，在数据库内部运行或与数据库紧密结合，从而提高对磁流体动力学湍流的理解，这将对整个天体物理学产生广泛的影响;新的方法和具体的工具将影响大多数科学领域。

AST-0428325VishniacCutting-edge science has become extremely data-intensive, and therefore the inability to store and analyze multi-Terabyte datasets today (and Petabytes tomorrow) is increasingly limiting scientific progress. One important, long-standing problem, described by Feynman as the last unsolved problem in classical physics, is turbulence in magnetized and unmagnetized fluids. A formidable obstacle is that the intrinsic physics of the turbulent cascade takes place in a Lagrangian frame, so that fluid particles must be tracked backward in time. This requirement holds true in other fields, such as cosmogenesis, where galaxies need to be tracked back through past encounters.The present research project will develop a new approach to large-scale numerical simulations, saving almost every time step in a database and thus permitting subsequent analysis by relatively simple computation, only over a lot of data. The new idea, which has already been tested, is to avoid the bottleneck of moving data to compute nodes, and instead combine low-level, indexed data access with processing that occurs right inside the database. This method permits the intended novel assault on the problem of turbulence, combining a forefront science project with a demonstration of the simulation environment of the future. It requires the development of tools and procedures to organize the data in multiple formats, each appropriate for a particular kind of statistical analysis, running either inside or closely integrated to the database.The resultant improved understanding of magneto-hydrodynamic turbulence will have a broad impact throughout astrophysics; and the new approach and concrete tools to be created will affect most areas of science.