III: Small: Collaborative Research: Neural Volume Visualization

III：小：协作研究：神经体积可视化

• 批准号: 2007444
• 负责人: Matthew Berger
• 金额: $ 30.05万
• 依托单位: Vanderbilt University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007444&HistoricalAwards=false
• 关键词: III; Small; Collaborative; Research; Neural

Data visualization is a key component to discovery for domain experts in a variety of scientific fields, ranging from atmospheric and ocean sciences, energy science, geosciences, and computational chemistry. Within these domains, visualization techniques for 3D volumetric data are commonly used to understand datasets produced by computational simulations. In turn, the insights gained through volume visualization are used to inform an expert on new simulations to run, leading to a cycle of visual analysis and simulation that supports a domain expert's workflow. Yet, this cycle is often impeded by the sheer size and complexity of the data, typified as high spatial resolution, time-varying, and multivariate, leading to two main problems. First, there are practical limitations to data access. Simulations are typically run on high-performance computing clusters, thus it is time and memory consuming to transfer data from these computational resources. Secondly, it is challenging to understand relationships between volumes across the aforementioned space of parameters. This project will address these problems through the development of deep learning-based volume visualization techniques that are compressive, interactive, trustworthy, and enable intuitive analysis. This research will study how now-standard volume visualization techniques can be decomposed into learnable components and fixed-function visualization operations, producing surrogate visualization models that will support experts across a variety of domains by facilitating visual analysis, and improving the discovery of relationships within complex datasets. This project will also hold local workshops for training graduate students across different fields to use the developed visualization methods in their respective domains.The development of surrogate models for volume visualization represents a new perspective on how to use machine learning for data visualization, where such models will be designed to reason about visualization processes, namely volume rendering and isosurfacing. This project will consider a full design space for learning surrogate models from different aspects of volume visualization, namely (1) learned volumetric representations such as function-space neural networks and volumetric feature embeddings, (2) parameters of visualization processes such as transfer functions and isovalues, and (3) the visualization process itself such as image formation in volume rendering or surface creation in isosurfacing. Furthermore, these surrogates will generalize to temporal sequences, multivariate volumes, and ensembles of volume simulations. Additionally, new techniques will be developed to utilize these surrogate models to improve volume visualization in three scenarios. Specifically the project will (1) inform end users about the trustworthiness of using machine learning for volume visualization, (2) learn simpler visual interfaces for user interaction, and finally (3) utilize latent representations of simulation factors for exploring complex relationships. The insights of this work go beyond just volume visualization, and they will offer new approaches to couple machine learning with visualization as a whole. This project will also disseminate the results, in the form of data used to train models, the models themselves, and software for both model training and visual exploration. Further, the investigators will collaborate with domain experts at their respective institutions to validate the quality, usability, and trustworthiness of the surrogate models, as well as translate the developed research into the practice of these domain experts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据可视化是从大气和海洋科学、能源科学、地球科学和计算化学等各种科学领域的领域专家发现数据的关键组成部分。在这些领域中，三维体数据的可视化技术通常用于理解由计算模拟产生的数据集。反过来，通过体积可视化获得的洞察力被用来通知专家要运行的新模拟，从而导致支持领域专家的工作流的视觉分析和模拟的循环。然而，这一周期常常被数据的巨大规模和复杂性所阻碍，表现为高空间分辨率、时变性和多变量，这导致了两个主要问题。首先，数据访问存在实际限制。模拟通常在高性能计算集群上运行，因此从这些计算资源传输数据是耗费时间和内存的。其次，要理解上述参数空间中体积之间的关系是具有挑战性的。这个项目将通过开发基于深度学习的体积可视化技术来解决这些问题，这些技术是压缩的、交互的、值得信赖的，并能够进行直观的分析。这项研究将研究如何将现在的标准体可视化技术分解为可学习的组件和固定功能的可视化操作，产生替代可视化模型，通过促进视觉分析和改进发现复杂数据集中的关系来支持各种领域的专家。该项目还将举办本地研讨会，培训不同领域的研究生在各自领域使用开发的可视化方法。体可视化代理模型的开发代表了如何使用机器学习进行数据可视化的新视角，其中此类模型将设计为对可视化过程进行推理，即体绘制和等值面。本项目将考虑一个完整的设计空间，用于从体积可视化的不同方面学习代理模型，即(1)学习的体积表示，如函数空间神经网络和体积特征嵌入，(2)可视化过程的参数，如传递函数和等值线，以及(3)可视化过程本身，如体积绘制中的图像形成或等值面中的表面创建。此外，这些替代物将推广到时间序列、多变量体积和体积模拟的集合。此外，将开发新的技术来利用这些代理模型来改进三种情况下的体可视化。具体地说，该项目将(1)向最终用户告知使用机器学习进行体积可视化的可靠性，(2)学习用于用户交互的更简单的可视界面，以及(3)利用模拟因素的潜在表示来探索复杂的关系。这项工作的洞察力不仅限于体积可视化，他们还将提供将机器学习与可视化作为一个整体进行结合的新方法。该项目还将以用于训练模型的数据、模型本身以及用于模型训练和视觉探索的软件的形式传播结果。此外，调查人员将与各自机构的领域专家合作，验证代理模型的质量、可用性和可信度，并将发展的研究转化为这些领域专家的实践。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Neural Flow Map Reconstruction

• DOI: 10.1111/cgf.14549
• 发表时间: 2022-06
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Saroj Sahoo;Y. Lu;M. Berger

PRAGMA: Interactively Constructing Functional Brain Parcellations

• DOI: 10.1109/vis47514.2020.00016
• 发表时间: 2020-09
• 期刊: 2020 IEEE Visualization Conference (VIS)
• 作者: Roza G. Bayrak;Nhung Hoang;Colin B. Hansen;Catie Chang;M. Berger

Compressive Neural Representations of Volumetric Scalar Fields

• DOI: 10.1111/cgf.14295
• 发表时间: 2021-04
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: Yuzhe Lu;K. Jiang;J. Levine;M. Berger

Interactively Assessing Disentanglement in GANs

• DOI: 10.1111/cgf.14524
• 发表时间: 2022-06
• 期刊: Computer Graphics Forum
• 影响因子: 2.5
• 作者: S. Jeong;Shusen Liu;Matthew Berger