Visual-Haptic Perceptual Illusions for Hand-based Interaction in Virtual Environments

虚拟环境中基于手的交互的视觉-触觉感知错觉

• 批准号: 450247716
• 负责人: Professor Dr. Antonio Krüger
• 金额: --
• 依托单位: Arbeitsbereich Human-Computer Interaction (HCI)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/450247716?language=en
• 关键词: Visual; Haptic; Perceptual; Illusions; Hand

Virtual Reality (VR) enables users to experience computer-generated environments and digital data in a natural and immersive way, in particular, by allowing users to directly grab virtual objects with their hands. Unfortunately, most current VR experiences still lack realistic haptic feedback, which hinders that users can truly ‘feel’ the virtual environment (VE) including tactile, kinesthetic, proprioceptive and vestibular cues. However, during hand-based interactions, users do not only rely on haptic and tactile feedback, but instead multisensory cues from vision, feeling and hearing are combined during the multisensory integration process. When senses are in conflict, the human brain weights different stimuli according to their reliability. Since visual perception often dominates during this integration process, visual illusions can be used in order to alter the haptic perception, which is often referred to as crossmodal interactions. Such crossmodal interactions for hand-based VR interaction using active and passive haptics have the potential to provide a compelling high-fidelity multimodal feedback. However, basic research is required to fully understand the crossmodal interplay between vision and haptics for human-computer interaction (HCI) in VR.In this proposed research project, we aim to investigate the interplay between vision and haptics to provide novel forms of visual-haptic hand-based VR interaction. First, we aim to investigate the effects of visual dominance and perceptual illusions to alter interaction with virtual objects. For this, we study techniques that actively modify how the user visually perceives the VE. For example, with redirected touching the user's virtual hands are displaced with an offset from the real hand positions to modify the user's movement towards or across physical proxy surfaces. In the second phase, we focus on how passive proxy design and active feedback can enhance haptic-based VR interactions. We plan to explore the space of visual-haptic feedback for VR spanned by the active-passive haptics continuum and an active-passive visual feedback dimension. In this context, we aim to investigate how actuators, active vibration feedback, passive haptic props and specific visualizations can enhance interaction and perception in VEs. Finally, we will provide an open-source framework including manufacturing schematics and instructions for the audience interested in building their own VEs supporting visual-haptic perceptual illusions.

虚拟现实(VR)使用户能够以自然和身临其境的方式体验计算机生成的环境和数字数据，特别是允许用户直接用手抓取虚拟对象。遗憾的是，目前的大多数VR体验仍然缺乏逼真的触觉反馈，这阻碍了用户真正地感受到虚拟环境(VE)，包括触觉、动觉、本体感觉和前庭提示。然而，在基于手的交互中，用户不仅依赖于触觉和触觉反馈，而是在多感觉整合过程中结合了来自视觉、感觉和听觉的多感觉线索。当感觉发生冲突时，人脑会根据刺激的可靠性对不同的刺激进行加权。由于视觉感知在整合过程中往往占主导地位，因此可以使用视觉错觉来改变触觉感知，这通常被称为交叉模式交互作用。这种使用主动和被动触觉的基于手的VR交互的跨模式交互具有提供引人注目的高保真多模式反馈的潜力。然而，要完全理解VR中人机交互(HCI)中视觉和触觉之间的交叉模式相互作用，还需要进行基础研究。在这项拟议的研究项目中，我们的目标是研究视觉和触觉之间的相互作用，以提供基于视觉和触觉的手虚拟现实交互的新形式。首先，我们的目标是研究视觉优势和知觉错觉对改变与虚拟物体的交互的影响。为此，我们研究了主动修改用户对VE的视觉感知方式的技术。例如，通过重定向触摸，用户的虚拟手以与真实手位置的偏移量移位，以修改用户朝向或穿过物理代理表面的移动。在第二阶段，我们重点研究被动代理设计和主动反馈如何增强基于触觉的VR交互。我们计划探索虚拟现实的视觉-触觉反馈空间，跨越主动-被动触觉连续体和主动-被动视觉反馈维度。在这种背景下，我们的目标是研究致动器、主动振动反馈、被动触觉道具和特定的可视化如何增强VES中的交互和感知。最后，我们将为有兴趣构建自己的VE支持视觉触觉错觉的观众提供一个开源框架，包括制造原理图和说明。