Visual-Haptic Perceptual Illusions for Hand-based Interaction in Virtual Environments

虚拟现实（VR）使用户能够以自然和身临其境的方式体验计算机生成的环境和数字数据，特别是允许用户直接用手抓住虚拟物体。不幸的是，目前大多数VR体验仍然缺乏真实的触觉反馈，这阻碍了用户真正“感受”虚拟环境（VE），包括触觉、动觉、本体感受和前庭提示。然而，在基于手的交互过程中，用户不仅仅依赖于触觉和触觉反馈，而是在多感觉整合过程中将视觉、感觉和听觉的多感官线索结合在一起。当感觉冲突时，人类的大脑会根据它们的可靠性来权衡不同的刺激。由于在这一整合过程中视觉感知通常占主导地位，因此可以使用视觉错觉来改变触觉感知，这通常被称为跨模态相互作用。这种使用主动和被动触觉的基于手的VR交互的跨模态交互有可能提供令人信服的高保真多模态反馈。然而，要充分了解虚拟现实中人机交互（HCI）中视觉和触觉之间的跨模态相互作用，还需要进行基础研究。在本研究项目中，我们的目标是研究视觉和触觉之间的相互作用，以提供基于手的视觉-触觉VR交互的新形式。首先，我们的目标是研究视觉优势和知觉错觉对改变与虚拟物体交互的影响。为此，我们研究了积极改变用户视觉感知VE方式的技术。例如，通过重定向触摸，用户的虚拟手与真实手的位置发生偏移，以修改用户朝向或跨越物理代理表面的移动。在第二阶段，我们关注被动代理设计和主动反馈如何增强基于触觉的VR交互。我们计划探索由主-被动触觉连续体和主-被动视觉反馈维度跨越的VR视觉-触觉反馈空间。在此背景下，我们的目标是研究致动器、主动振动反馈、被动触觉道具和特定可视化如何增强虚拟现实中的交互和感知。最后，我们将提供一个开源框架，包括制造原理图和说明，为有兴趣建立自己的虚拟机支持视觉触觉感知错觉的观众。

Virtual Reality (VR) enables users to experience computer-generated environments and digital data in a natural and immersive way, in particular, by allowing users to directly grab virtual objects with their hands. Unfortunately, most current VR experiences still lack realistic haptic feedback, which hinders that users can truly ‘feel’ the virtual environment (VE) including tactile, kinesthetic, proprioceptive and vestibular cues. However, during hand-based interactions, users do not only rely on haptic and tactile feedback, but instead multisensory cues from vision, feeling and hearing are combined during the multisensory integration process. When senses are in conflict, the human brain weights different stimuli according to their reliability. Since visual perception often dominates during this integration process, visual illusions can be used in order to alter the haptic perception, which is often referred to as crossmodal interactions. Such crossmodal interactions for hand-based VR interaction using active and passive haptics have the potential to provide a compelling high-fidelity multimodal feedback. However, basic research is required to fully understand the crossmodal interplay between vision and haptics for human-computer interaction (HCI) in VR.In this proposed research project, we aim to investigate the interplay between vision and haptics to provide novel forms of visual-haptic hand-based VR interaction. First, we aim to investigate the effects of visual dominance and perceptual illusions to alter interaction with virtual objects. For this, we study techniques that actively modify how the user visually perceives the VE. For example, with redirected touching the user's virtual hands are displaced with an offset from the real hand positions to modify the user's movement towards or across physical proxy surfaces. In the second phase, we focus on how passive proxy design and active feedback can enhance haptic-based VR interactions. We plan to explore the space of visual-haptic feedback for VR spanned by the active-passive haptics continuum and an active-passive visual feedback dimension. In this context, we aim to investigate how actuators, active vibration feedback, passive haptic props and specific visualizations can enhance interaction and perception in VEs. Finally, we will provide an open-source framework including manufacturing schematics and instructions for the audience interested in building their own VEs supporting visual-haptic perceptual illusions.