III: Small: Collaborative Research: Neural Volume Visualization

点击翻译按钮获取中文摘要

Data visualization is a key component to discovery for domain experts in a variety of scientific fields, ranging from atmospheric and ocean sciences, energy science, geosciences, and computational chemistry. Within these domains, visualization techniques for 3D volumetric data are commonly used to understand datasets produced by computational simulations. In turn, the insights gained through volume visualization are used to inform an expert on new simulations to run, leading to a cycle of visual analysis and simulation that supports a domain expert's workflow. Yet, this cycle is often impeded by the sheer size and complexity of the data, typified as high spatial resolution, time-varying, and multivariate, leading to two main problems. First, there are practical limitations to data access. Simulations are typically run on high-performance computing clusters, thus it is time and memory consuming to transfer data from these computational resources. Secondly, it is challenging to understand relationships between volumes across the aforementioned space of parameters. This project will address these problems through the development of deep learning-based volume visualization techniques that are compressive, interactive, trustworthy, and enable intuitive analysis. This research will study how now-standard volume visualization techniques can be decomposed into learnable components and fixed-function visualization operations, producing surrogate visualization models that will support experts across a variety of domains by facilitating visual analysis, and improving the discovery of relationships within complex datasets. This project will also hold local workshops for training graduate students across different fields to use the developed visualization methods in their respective domains.The development of surrogate models for volume visualization represents a new perspective on how to use machine learning for data visualization, where such models will be designed to reason about visualization processes, namely volume rendering and isosurfacing. This project will consider a full design space for learning surrogate models from different aspects of volume visualization, namely (1) learned volumetric representations such as function-space neural networks and volumetric feature embeddings, (2) parameters of visualization processes such as transfer functions and isovalues, and (3) the visualization process itself such as image formation in volume rendering or surface creation in isosurfacing. Furthermore, these surrogates will generalize to temporal sequences, multivariate volumes, and ensembles of volume simulations. Additionally, new techniques will be developed to utilize these surrogate models to improve volume visualization in three scenarios. Specifically the project will (1) inform end users about the trustworthiness of using machine learning for volume visualization, (2) learn simpler visual interfaces for user interaction, and finally (3) utilize latent representations of simulation factors for exploring complex relationships. The insights of this work go beyond just volume visualization, and they will offer new approaches to couple machine learning with visualization as a whole. This project will also disseminate the results, in the form of data used to train models, the models themselves, and software for both model training and visual exploration. Further, the investigators will collaborate with domain experts at their respective institutions to validate the quality, usability, and trustworthiness of the surrogate models, as well as translate the developed research into the practice of these domain experts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.