Maximizing the VR experience : testing new ways to integrate 3D sound and eye-tracking

该Engage项目旨在通过在StellarX平台内实现自然3D声音，为增强虚拟现实(VR)系统做出贡献，以培训参与危机管理和应急响应的第一响应者之间的互操作性。VR的主要前提是创造一种交替性，但如果没有合适的音频提示来匹配视觉效果，大脑就不会相信这种错觉。最近的研究表明，在虚拟现实环境中添加3D声音可以提供更好的临场感，可以提高习得和任务表现，并有助于将注意力重新定向到相关方面。我们的方法结合了实验心理学和认知神经人体工程学，将OVA的StellarX平台变成了一个优化了自然3D声音的强大训练系统。这将通过使用双耳录音技术录制一组与虚拟环境相匹配的声音来实现，以便准确地捕捉来自各个方向的声音信息并产生极其逼真的录音。此外，我们计划验证虚拟现实中与自然3D声音相关的有益方面是否可以在联合互操作性培训的背景下观察到。这将通过评估临场感、技能获得和任务表现，以及受训者将注意力重新引导到相关3D声音的能力来完成。结合主观和客观(行为和视觉)指标来评估这些收益代表了该项目的一项关键创新。结果和结果将为组建工作人员提供先进的虚拟现实培训方法和更好的工具，从而允许采用有针对性的技术来提高个人或团队的表现。此外，这种类型的培训将改善学习，因为受训者在模拟中的临场感将通过视觉和听觉沉浸得到最大化，并且模拟内容可以很容易地适应特定需求。因此，该项目将对加拿大科技、金融和教育部门以及公安、智能家教和游戏行业等其他领域产生积极影响。

This Engage project aims to contribute to the enhancement of a virtual reality (VR) system for traininginteroperability between first responders involved in crisis management and emergency response byimplementing natural 3D sound within the StellarX platform. The main premise of VR is to create an alternatereality, but without the right audio cues to match the visuals, the brain does not buy into the illusion. Recentstudies indicate that adding 3D sound to a VR environment offers a greater sense of presence, can improveskills acquisition and task performance, and facilitate attention redirection to relevant aspects. Our approachcombines experimental psychology and cognitive neuroergonomics to turn OVA's platform StellarX into apowerful training system optimized with natural 3D sound. This will be done by recording a set of sounds thatmatch the virtual environment using a binaural recording technique in order to accurately capture sonicinformation coming from all directions and produce extremely realistic recordings. In addition, we plan toverify whether the beneficial aspects associated with natural 3D sound in VR can be observed in a jointinteroperability-training context. This will be done by assessing the sense of presence, skills acquisition, andtask performance, as well as trainees' capacity to redirect attention to relevant 3D sounds. The combination ofsubjective and objective (behavioral and ocular) metrics to evaluate these benefits represents a key innovationof this project. Results and outcomes will provide an advanced VR training method and a better tool for theformation of staff, thus allowing targeted techniques to be employed to improve individual or teamperformance. Moreover, this type of training will improve learning since the trainee's sense of presence in thesimulation will be maximized through visual and aural immersion and the simulation content can easily beadapted to reflect specific needs. Therefore, this project will positively influence the Canadian technological,financial and education sectors, as well as other areas such as public security, intelligent tutoring and thegaming industry.