Study and development of immersive multi-sensory systems for avatar embodiment in virtual reality

沉浸式虚拟现实（VR）不仅提供了模拟用户周围环境的可能性，而且还提供了他的视觉自我表现。换句话说，用户将自己视为化身，而不是看到他的实际身体。当化身以非常短的延迟模仿用户的移动时，结果是化身的所有权的主观感觉，其中用户感觉化身是他的实际物理身体。当虚拟身体不与物理身体对齐时，用户体验到本体感受漂移，其中他的肢体的感知位置朝向虚拟化身的位置漂移。近年来，我们的团队通过开发一个VR系统来利用这种错觉，该系统已被证明可以通过实时操纵用户化身的动作来改变用户的动作。我们已经发现，用户的大脑信号可以用来测量所有权的错觉，并且可以使用振动器来产生错觉，振动器产生运动的错觉，而不是实际的运动。 我们最近的进展使我们能够在此基础上将我们的研究计划提升到一个新的水平，以探索和应用实时化身的潜力，朝着我们雄心勃勃的长期目标发展，即开发一个由用户控制和感受的完整虚拟代理身体。本研究计划由三个方面组成。第一个目标是将我们目前的技术整合和扩展到一个由运动或想象运动控制的多感官虚拟身体中，并通过本体感受错觉来感受。第二个目标是开发一个系统，允许化身的上肢和下肢逐渐实时变形（单侧长度、周长、段比例等）以修改用户的身体自我表征并改变运动控制。第三个目标是将触觉锻炼器与我们的自我化身结合起来，联合收割机。在这里，化身将由用户产生的运动和力量驱动，以恢复中风后患者的视觉和运动控制之间的一致性。 这项研究将导致完全体现化身，将用于身体康复（研究和治疗导致运动障碍的病理，如中风和脊髓损伤），体育（改善运动生物力学和学习新任务），远程呈现系统（允许远程用户感觉完全存在），机器人技术（感觉被控制的机器人的运动）和全身代理（允许瘫痪患者控制虚拟或机器人身体）。该研究项目将培训4名研究生和4名本科生，使他们具备加拿大工业界最前沿技术和高需求的技能。他们将学习如何：开发复杂的多感官VR系统;开发高度互动的视频游戏和实时计算机动画;获取、处理和解释生物医学信号;开发新颖的人机交互（HCI）方法。

Immersive virtual reality (VR) offers the possibility of simulating not only a user's surroundings but also his visual self-representation. In other words, the user sees himself as an avatar rather than seeing his actual body. When the avatar mimics the user's movements with a very short delay, the result is a subjective feeling of ownership of the avatar where the user feels the avatar is his actual physical body. When the virtual body is not aligned with the physical body, users experience a proprioceptive drift where the perceived position of his limbs drifts towards that of the virtual avatar. In recent years, our team has harnessed this illusion through the development of a VR system that has been shown to alter a user's movements through the real-time manipulation of the movements of his avatar. We have found that the user's brain signals can be used to measure the illusion of ownership and that the illusion can be created using vibrators that give the illusion of movement, rather than actual movements. Our recent progress lead us to build upon this and take our research program to a new level in order to explore and apply the potential of real-time embodied avatars, progressing towards our ambitious long-term goal of developing a complete virtual surrogate body that is controlled and felt by its user. This research program consists of three threads. The 1st aims to integrate and extend our current technologies into a multisensory virtual body controlled by movement or imagined movements and felt through proprioceptive illusions. The 2nd aims to develop a system that allows for gradual real-time deformation of the avatar's upper and lower limbs (unilateral length, girth, segment proportions, etc.) during the embodiment experience in order to modify the user's body self-representation and alter motor control. The 3rd aims to combine a haptic exerciser with our self-avatar. Here the avatar will be driven by the movements and forces produced by users, in order to restore congruency between vision and motor control in post-stroke patients. This research will lead to fully-embodied avatars that will be used in physical rehabilitation (to study and treat pathologies that cause motor impairment, such as stroke and spinal cord injury), sports (improving sport biomechanics and learning new tasks), telepresence systems (allowing a remote user to feel fully present), robotics (feeling the movements of the robot that is controlled) and full-body surrogacy (allowing paralyzed patients to control a virtual or robotic body). The research program will train 4 graduate students and 4 undergraduates, equipping them with skills that are at the cutting edge of technology and in high demand in Canadian industry. They will learn how to: develop complex multisensory VR systems; develop highly interactive video games and real-time computer animations; acquire, process and interpret biomedical signals; develop novel human-computer interaction (HCI) methods.