BIGDATA: Small: DA: Data Summarization, Analysis, and Triage for Very Large Scale Flow Fields

经常生成大数据集的计算科学家面临两个主要挑战。 第一个是决定哪些数据对于分析是最重要的，因为只有一小部分数据可以保留。 第二是将这些数据转化为传达最深入见解的信息。 随着模拟输出的规模不断增长，“先保存数据，后分析”的方法需要完全替换为更积极的数据优先级排序和减少，然后才能进行任何分析。 在这个项目中，开发了核心数据分析技术，以促进大规模流量数据的有效数据汇总，索引和分类。流体流动在解释许多学科中的许多现象方面起着重要作用。 为了向科学家提供数据内容的简洁视图，并根据其相似性和复杂性组织数据和特征，开发了基于图形的模型，以同时揭示流场的主要结构，并促进高性能和核心外流线计算。我们开发的流线有效地组和优先级的子区域的统计和几何复杂性的措施，以允许有效的数据访问。为了表征流场的时间复杂性，我们开发了随时间变化的分析算法，可以更详细地分析数据，并为用户提供灵活的界面，以快速识别显着feature.The开发的综合流分析和可视化框架最初针对两个应用程序，涡轮机械的空气动力学模拟，和研究马登朱利安振荡在气候建模。由于涡轮机械中的典型流动充满了不断变化的冲击和旋涡结构，可视化允许设计人员在相对较短的时间内识别损失区域和复杂的流动特征，如果这些特征可以自动识别的话。 为了理解马登朱利安振荡现象，因为这种现象与空气对流密切相关，在本项目下开发的流动分析技术可用于识别和跟踪其位置和持续时间。由于时变模拟生成的数据大小可能非常大，因此所提出的时变数据简化技术允许科学家专注于数据的最突出部分。 该项目的关键影响是提供一个有效且有吸引力的解决方案，以帮助科学家理解大规模模拟生成的大量数据。通过与应用科学家的密切合作，该项目中开发的研究思路将转化为开源软件框架。

Two major challenges are faced by computational scientists who routinely generate big data sets. The first is deciding what data are the most essential for analysis, given that only a small fraction of them can be retained. The second is transforming these data into information that conveys the most insight. As the size of simulation output continues to grow, the "save the data first, analyze them later" approach needs to be completely replaced with more aggressive data prioritization and reduction before any analysis can be done. In this project, core data analytics technologies are developed to facilitate effective data summarization, indexing, and triage for large-scale flow data. Fluid flow plays an important role in explaining many phenomena across a wide range of disciplines. To provide the scientists with a succinct view of the data content, and also organize the data and features based on their similarity and complexity, a graph-based model is developed to simultaneously reveal the major structure of the flow field, and to facilitate high performance and out-of-core flow line computation. We develop statistical and geometrical complexity measures for the flow lines to efficiently group and prioritize sub-regions in the vector field to allow efficient data access. To characterize the temporal complexity of flow fields, we develop time-varying analysis algorithms that allow for more detailed analysis of the data, and provide the user with flexible interface to quickly identify salient features.The development of the proposed integrated flow analysis and visualization framework initially targeted two applications, simulations of turbo machinery in aerodynamics, and study of Madden Julian Oscillation in climate modeling. As typical flow in turbo machinery is full of evolving shocks and vortical structures, visualization allows the designers to identify loss regions and complex flow features in a relatively short amount of time if these features can be identified automatically. To understand the phenomenon of Madden Julian Oscillation, as this phenomenon is strongly related to the convection of air, the flow analysis techniques developed under this project can be used to identify and track its locations and durations. Because the size of data generated by time-varying simulations can be prohibitively large, the proposed time-varying data reduction techniques allow scientists to focus on the most salient portion of the data. The key impact of this project is to make available a working and attractive solution to assist scientists to comprehend the vast amount of data generated by large-scale simulations. Through close collaboration with application scientists, the research ideas developed in this project into will be transformed into an open source software framework.