Study and development of immersive multi-sensory systems for avatar embodiment in virtual reality

虚拟现实中化身体现的沉浸式多感官系统的研究和开发

• 批准号: RGPIN-2020-06171
• 负责人: Labbe, David
• 金额: $ 2.11万
• 依托单位: École de technologie supérieure
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=754240
• 关键词: Study; development; immersive; multi; sensory

Immersive virtual reality (VR) offers the possibility of simulating not only a user's surroundings but also his visual self-representation. In other words, the user sees himself as an avatar rather than seeing his actual body. When the avatar mimics the user's movements with a very short delay, the result is a subjective feeling of ownership of the avatar where the user feels the avatar is his actual physical body. When the virtual body is not aligned with the physical body, users experience a proprioceptive drift where the perceived position of his limbs drifts towards that of the virtual avatar. In recent years, our team has harnessed this illusion through the development of a VR system that has been shown to alter a user's movements through the real-time manipulation of the movements of his avatar. We have found that the user's brain signals can be used to measure the illusion of ownership and that the illusion can be created using vibrators that give the illusion of movement, rather than actual movements. Our recent progress lead us to build upon this and take our research program to a new level in order to explore and apply the potential of real-time embodied avatars, progressing towards our ambitious long-term goal of developing a complete virtual surrogate body that is controlled and felt by its user. This research program consists of three threads. The 1st aims to integrate and extend our current technologies into a multisensory virtual body controlled by movement or imagined movements and felt through proprioceptive illusions. The 2nd aims to develop a system that allows for gradual real-time deformation of the avatar's upper and lower limbs (unilateral length, girth, segment proportions, etc.) during the embodiment experience in order to modify the user's body self-representation and alter motor control. The 3rd aims to combine a haptic exerciser with our self-avatar. Here the avatar will be driven by the movements and forces produced by users, in order to restore congruency between vision and motor control in post-stroke patients. This research will lead to fully-embodied avatars that will be used in physical rehabilitation (to study and treat pathologies that cause motor impairment, such as stroke and spinal cord injury), sports (improving sport biomechanics and learning new tasks), telepresence systems (allowing a remote user to feel fully present), robotics (feeling the movements of the robot that is controlled) and full-body surrogacy (allowing paralyzed patients to control a virtual or robotic body). The research program will train 4 graduate students and 4 undergraduates, equipping them with skills that are at the cutting edge of technology and in high demand in Canadian industry. They will learn how to: develop complex multisensory VR systems; develop highly interactive video games and real-time computer animations; acquire, process and interpret biomedical signals; develop novel human-computer interaction (HCI) methods.

沉浸式虚拟现实(VR)不仅提供了模拟用户周围环境的可能性，还提供了模拟用户视觉自我表征的可能性。换句话说，用户将自己视为化身，而不是看到自己的实际身体。当化身以非常短的延迟模仿用户的动作时，结果是化身的主观感，其中用户感觉化身就是他实际的身体。当虚拟身体与物理身体不对齐时，用户体验本体感觉漂移，其中他的四肢的感知位置向虚拟化身的位置漂移。近年来，我们的团队通过开发一种VR系统来利用这种错觉，该系统被证明可以通过实时操纵用户化身的动作来改变用户的动作。我们发现，用户的大脑信号可以用来衡量所有权的错觉，这种错觉可以使用给人一种运动错觉的振动器来创造，而不是实际的运动。我们最近的进展使我们在此基础上再接再厉，并将我们的研究计划提升到一个新的水平，以探索和应用实时虚拟化身的潜力，朝着我们雄心勃勃的长期目标前进，即开发一个由用户控制和感受的完整虚拟代理体。这项研究计划由三条主线组成。第一个目标是将我们目前的技术整合和扩展到一个由运动或想象的运动控制的多感官虚拟身体中，并通过本体感觉错觉感受到。第二个目的是开发一种系统，允许化身的上肢和下肢(单侧长度、周长、节段比例等)的逐渐实时变形。在实施过程中体验以修改用户的身体自我表征和改变运动控制。第三个目标是将触觉训练器与我们的自我化身结合起来。在这里，化身将由用户产生的运动和力量驱动，以恢复中风后患者的视觉和运动控制之间的一致性。这项研究将带来全实体化身，这些化身将用于物理康复(研究和治疗导致运动损伤的病理，如中风和脊髓损伤)、运动(改善运动生物力学和学习新任务)、远程呈现系统(使远程用户感觉完全在场)、机器人(感受受控机器人的运动)和全身代孕(允许瘫痪患者控制虚拟或机器人身体)。该研究计划将培养4名研究生和4名本科生，使他们掌握加拿大工业对尖端技术和高需求的技能。他们将学习如何：开发复杂的多感官虚拟现实系统；开发高度互动的视频游戏和实时计算机动画；获取、处理和解释生物医学信号；开发新的人机交互(HCI)方法。