Maximizing the VR experience : testing new ways to integrate 3D sound and eye-tracking

点击翻译按钮获取中文摘要

This Engage project aims to contribute to the enhancement of a virtual reality (VR) system for traininginteroperability between first responders involved in crisis management and emergency response byimplementing natural 3D sound within the StellarX platform. The main premise of VR is to create an alternatereality, but without the right audio cues to match the visuals, the brain does not buy into the illusion. Recentstudies indicate that adding 3D sound to a VR environment offers a greater sense of presence, can improveskills acquisition and task performance, and facilitate attention redirection to relevant aspects. Our approachcombines experimental psychology and cognitive neuroergonomics to turn OVA's platform StellarX into apowerful training system optimized with natural 3D sound. This will be done by recording a set of sounds thatmatch the virtual environment using a binaural recording technique in order to accurately capture sonicinformation coming from all directions and produce extremely realistic recordings. In addition, we plan toverify whether the beneficial aspects associated with natural 3D sound in VR can be observed in a jointinteroperability-training context. This will be done by assessing the sense of presence, skills acquisition, andtask performance, as well as trainees' capacity to redirect attention to relevant 3D sounds. The combination ofsubjective and objective (behavioral and ocular) metrics to evaluate these benefits represents a key innovationof this project. Results and outcomes will provide an advanced VR training method and a better tool for theformation of staff, thus allowing targeted techniques to be employed to improve individual or teamperformance. Moreover, this type of training will improve learning since the trainee's sense of presence in thesimulation will be maximized through visual and aural immersion and the simulation content can easily beadapted to reflect specific needs. Therefore, this project will positively influence the Canadian technological,financial and education sectors, as well as other areas such as public security, intelligent tutoring and thegaming industry.