CHS: Medium: Collaborative Research: Sculpting Visualizations: Toward a Practice and Theory of 3D Scientific Visualizations Using Physical Objects and Augmented Reality

CHS：媒介：协作研究：雕刻可视化：利用物理对象和增强现实实现 3D 科学可视化的实践和理论

• 批准号: 1704604
• 负责人: Daniel Keefe
• 金额: $ 67.5万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1704604&HistoricalAwards=false
• 关键词: CHS; Medium; Collaborative; Research; Sculpting

Advances in 3D printing, augmented reality (AR), and virtual reality (VR), are creating new possibilities for computing tools that integrate with our physical environment and take advantage of our physical abilities. This proposal will study how these technologies can support 3D data visualization, an increasingly important and common scientific activity. The team, which includes computer scientists, artists, neuroscientists, geologists, and oceanologists, will first use 3D printing to develop physical representations of 3D data ("physical data forms") that match the needs of specific analysis domains and tasks, and through this develop principles for doing 3D design for visualizations in general. They will then design hybrid spaces that use AR visual representations along with physical data forms, looking at ways to leverage the strengths of each and developing ways to interact with the data through both the virtual and physical forms. Finally, they will create tools that leverage these design principles and interaction techniques to allow scientists to create new physical data forms and hybrid visualizations to address outstanding data analysis challenges in brain imaging, geology, and, ultimately, many scientific fields. The work will support interdisciplinary courses at the intersection of art, science, computing, and data visualization at the PIs' institutions; students will also be trained in research methods and work with the research team to develop public science museum exhibits that raise awareness of both the technology and the science involved. To leverage the possibilities of rapid, creative, artistic iteration and exploration of physical form, the team will develop interfaces and algorithms for capturing and extracting properties of physical forms, along with tools for exploring mappings between these properties and 3D data, a design theory and taxonomy, and a catalog for using physical inputs in visualization processes. These physical elements will be augmented with colocated digital head-tracked stereoscopic displays that directly incorporate the printed objects into the AR experience, along with touch-based interaction techniques such as touch-sensitive input surfaces or the direct inclusion of physical widgets in the printed objects. These visualizations will be evaluated through user studies based on existing methodologies for comparing 3D vector field visualization methods. The team will then develop exploratory visualization tools, using streamlined versions of the catalog and visualizations developed earlier to help manage the complexity of creating new visualizations while teaching visual design processes to scientists through the use of the tools, recasting the scientific task of data exploration as a creative process of visualization design to support learning, engagement, and effective analysis. These tools will be iteratively developed by teams of art and computer science students in conjunction with domain scientists and used to facilitate data exploration and discovery, as well as to bring science more directly into the public sphere through interactive experiences, such as at science museums.

3D打印、增强现实（AR）和虚拟现实（VR）的进步正在为计算工具创造新的可能性，这些工具与我们的物理环境相结合，并利用我们的物理能力。 该提案将研究这些技术如何支持3D数据可视化，这是一项日益重要和常见的科学活动。 该团队包括计算机科学家，艺术家，神经科学家，地质学家和海洋学家，将首先使用3D打印来开发3D数据的物理表示（“物理数据形式”），以满足特定分析领域和任务的需求，并通过此开发进行3D可视化设计的原则。 然后，他们将设计使用AR视觉表示沿着物理数据形式的混合空间，寻找利用两者优势的方法，并开发通过虚拟和物理形式与数据交互的方法。 最后，他们将创建工具，利用这些设计原则和交互技术，使科学家能够创建新的物理数据形式和混合可视化，以解决大脑成像，地质学以及最终许多科学领域的突出数据分析挑战。 这项工作将支持PI机构的艺术，科学，计算和数据可视化交叉的跨学科课程;学生还将接受研究方法的培训，并与研究团队合作开发公共科学博物馆展品，提高对所涉及的技术和科学的认识。为了利用快速、创造性、艺术性的迭代和物理形式探索的可能性，该团队将开发用于捕获和提取物理形式属性的接口和算法，沿着用于探索这些属性与3D数据之间映射的工具，设计理论和分类法，以及用于在可视化过程中使用物理输入的目录。 这些物理元素将通过共位数字头部跟踪立体显示器来增强，该显示器直接将打印对象并入AR体验中，沿着基于触摸的交互技术，例如触摸敏感输入表面或直接将物理小部件包含在打印对象中。 这些可视化将通过基于现有方法的用户研究进行评估，以比较3D矢量场可视化方法。 然后，该团队将开发探索性可视化工具，使用精简版本的目录和早期开发的可视化来帮助管理创建新可视化的复杂性，同时通过使用这些工具向科学家教授可视化设计过程，将数据探索的科学任务重新塑造为可视化设计的创造性过程，以支持学习，参与和有效分析。 这些工具将由艺术和计算机科学专业的学生团队与领域科学家一起迭代开发，用于促进数据探索和发现，并通过互动体验（如科学博物馆）将科学更直接地带入公共领域。

项目成果

AFFECTIVE PALETTES FOR SCIENTIFIC VISUALIZATION: GROUNDING ENVIRONMENTAL DATA IN THE NATURAL WORLD

科学可视化的情感调色板：将环境数据扎根于自然世界

• DOI: 10.1109/visap52981.2021.00009
• 发表时间: 2021
• 期刊: 2021 IEEE Vis
• 作者: Samsel, Francesca: Zeller;Abram, Greg;Keefe, Daniel
• 通讯作者: Keefe, Daniel

Printmaking, Puzzles, and Studio Closets: Using Artistic Metaphors to Reimagine the User Interface for Designing Immersive Visualizations

• DOI: 10.1109/visap51628.2020.00009
• 发表时间: 2020-10
• 期刊: 2020 IEEE VIS Arts Program (VISAP)
• 作者: Bridger Herman;F. Samsel;Annie Bares;Seth Johnson;G. Abram;Daniel F. Keefe

Multi-Touch Querying on Data Physicalizations in Immersive AR

• DOI: 10.1145/3488542
• 发表时间: 2021-11
• 期刊: Proceedings of the ACM on Human-Computer Interaction
• 作者: Bridger Herman;Maxwell Omdal;Stephanie Zeller;Clara A. Richter;F. Samsel;G. Abram;Daniel F. Keefe

Scientific Visualization: Enriching Vocabulary via the Human Hand

科学可视化：通过人手丰富词汇

• 发表时间: 2019
• 期刊: Proceedings of the IEEE VIS Arts Program (VISAP
• 作者: Samsel, Francesca;Johnson, Seth;Bares, Annie;Keefe, Daniel F.
• 通讯作者: Keefe, Daniel F.

Artifact-Based Rendering: Harnessing Natural and Traditional Visual Media for More Expressive and Engaging 3D Visualizations

• DOI: 10.1109/tvcg.2019.2934260
• 发表时间: 2020-01-01
• 期刊: IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
• 影响因子: 5.2
• 作者: Johnson, Seth;Samsel, Francesca;Keefe, Daniel F.
• 通讯作者: Keefe, Daniel F.