Visual-Haptic Perceptual Illusions for Hand-based Interaction in Virtual Environments

虚拟现实（VR）使用户能够以自然和身临其境的方式体验计算机生成的环境和数字数据，特别是通过允许用户直接用手抓住虚拟对象。不幸的是，大多数当前的VR体验仍然缺乏真实的触觉反馈，这阻碍了用户能够真正“感受”虚拟环境（VE），包括触觉、动觉、本体感受和前庭提示。然而，在基于手的交互过程中，用户不仅依赖于触觉和触觉反馈，而是在多感官整合过程中结合来自视觉、感觉和听觉的多感官提示。当感觉发生冲突时，人类大脑会根据不同刺激的可靠性对其进行加权。由于视觉感知通常在这个整合过程中占主导地位，因此可以使用视觉错觉来改变触觉感知，这通常被称为跨模态交互。使用主动和被动触觉的基于手的VR交互的这种跨模态交互具有提供引人注目的高保真多模态反馈的潜力。然而，基础研究需要充分了解视觉和触觉之间的跨模态相互作用的人机交互（HCI）在VR。在这个拟议的研究项目中，我们的目标是调查视觉和触觉之间的相互作用，以提供新的形式的视觉触觉手为基础的VR交互。首先，我们的目标是研究视觉优势和知觉错觉的影响，以改变与虚拟物体的互动。为此，我们研究的技术，积极修改用户的视觉感知的VE。例如，在重定向触摸的情况下，用户的虚拟手以与真实的手位置的偏移移位，以修改用户朝向或跨越物理代理表面的移动。在第二阶段，我们将重点关注被动代理设计和主动反馈如何增强基于触觉的VR交互。我们计划探索虚拟现实的视觉触觉反馈的空间跨越主动-被动触觉连续体和主动-被动视觉反馈维度。在这种情况下，我们的目标是研究如何致动器，主动振动反馈，被动触觉道具和特定的可视化可以增强虚拟环境中的互动和感知。最后，我们将提供一个开源的框架，包括制造原理图和指令，为有兴趣建立自己的虚拟环境支持视觉触觉感知错觉的观众。

Virtual Reality (VR) enables users to experience computer-generated environments and digital data in a natural and immersive way, in particular, by allowing users to directly grab virtual objects with their hands. Unfortunately, most current VR experiences still lack realistic haptic feedback, which hinders that users can truly ‘feel’ the virtual environment (VE) including tactile, kinesthetic, proprioceptive and vestibular cues. However, during hand-based interactions, users do not only rely on haptic and tactile feedback, but instead multisensory cues from vision, feeling and hearing are combined during the multisensory integration process. When senses are in conflict, the human brain weights different stimuli according to their reliability. Since visual perception often dominates during this integration process, visual illusions can be used in order to alter the haptic perception, which is often referred to as crossmodal interactions. Such crossmodal interactions for hand-based VR interaction using active and passive haptics have the potential to provide a compelling high-fidelity multimodal feedback. However, basic research is required to fully understand the crossmodal interplay between vision and haptics for human-computer interaction (HCI) in VR.In this proposed research project, we aim to investigate the interplay between vision and haptics to provide novel forms of visual-haptic hand-based VR interaction. First, we aim to investigate the effects of visual dominance and perceptual illusions to alter interaction with virtual objects. For this, we study techniques that actively modify how the user visually perceives the VE. For example, with redirected touching the user's virtual hands are displaced with an offset from the real hand positions to modify the user's movement towards or across physical proxy surfaces. In the second phase, we focus on how passive proxy design and active feedback can enhance haptic-based VR interactions. We plan to explore the space of visual-haptic feedback for VR spanned by the active-passive haptics continuum and an active-passive visual feedback dimension. In this context, we aim to investigate how actuators, active vibration feedback, passive haptic props and specific visualizations can enhance interaction and perception in VEs. Finally, we will provide an open-source framework including manufacturing schematics and instructions for the audience interested in building their own VEs supporting visual-haptic perceptual illusions.