Exploring the SenseMaking Process through Interactions and fNIRS in Immersive Visualization
Exploring the SenseMaking Process through Interactions and fNIRS in Immersive Visualization
复制标题
DOI：
10.1109/tvcg.2021.3067693
复制
发表时间：
2021-05-01
影响因子：
5.2
通讯作者：
Lu, Aidong
中科院分区：
文献类型：
--
作者：
Galati, Alexia;Schoppa, Riley;Lu, Aidong
在设计人机界面时，认知理论为我们的决策提供了信息，而沉浸式系统使我们能够检验这些理论。这项工作探讨了在沉浸式环境中的意义建构过程，通过研究内部和外部用户行为与一个经典的可视化问题：视觉比较和聚类任务。我们开发了一个沉浸式系统来进行用户研究，从不同的渠道收集用户行为数据：AR HMD用于捕获外部用户交互，功能近红外光谱（fNIRS）用于捕获内部神经序列，视频用于参考。为了检查意义构建，我们评估了界面的布局（平面2D与圆柱形3D布局）和任务的挑战水平（低与高认知负荷）如何影响用户的交互，这些交互如何随时间变化，以及它们如何影响任务表现。我们还开发了一个可视化系统来探索所有数据通道之间的联合模式。我们发现，增加相互作用和脑血流动力学反应与更准确的性能，特别是在认知要求的试验。布局类型并没有可靠地影响互动或任务性能。我们将讨论这些发现如何告知沉浸式系统的设计和评估，预测用户的性能和交互，并从体现和分布式认知的角度提供关于意义建构的理论见解。
Theories of cognition inform our decisions when designing human-computer interfaces, and immersive systems enable us to examine these theories. This work explores the sensemaking process in an immersive environment through studying both internal and external user behaviors with a classical visualization problem: a visual comparison and clustering task. We developed an immersive system to perform a user study, collecting user behavior data from different channels: AR HMD for capturing external user interactions, functional near-infrared spectroscopy (fNIRS) for capturing internal neural sequences, and video for references. To examine sensemaking, we assessed how the layout of the interface (planar 2D vs. cylindrical 3D layout) and the challenge level of the task (low vs. high cognitive load) influenced the users' interactions, how these interactions changed over time, and how they influenced task performance. We also developed a visualization system to explore joint patterns among all the data channels. We found that increased interactions and cerebral hemodynamic responses were associated with more accurate performance, especially on cognitively demanding trials. The layout types did not reliably influence interactions or task performance. We discuss how these findings inform the design and evaluation of immersive systems, predict user performance and interaction, and offer theoretical insights about sensemaking from the perspective of embodied and distributed cognition.