Detection and prediction of cybersickness in virtual and mixed reality environments using wearables

点击翻译按钮获取中文摘要

Virtual and mixed reality (VR/MR) systems have burgeoned over the last couple of years with applications in healthcare, gaming, telepresence, and skills training, to name a few. Within the skills training sector, for example, police/law enforcement training is an important application domain which has seen increased adoption worldwide. In Canada, Public Safety and the Royal Canadian Mounted Police (RCMP) have invested millions of dollars to set up state-of-the-art VR/MR facilities to train the next generation of law enforcement agents. VR allows for different scenes, conditions, and maneuvers to be tested in one single physical location, thus not only reducing training costs, but better equipping trainees to handle unknowns. While the potential is there, existing VR/MR systems are known to induce motion sickness - known as cybersickness - on a large proportion of the trainees, especially females. As such, cybersickness detection and prevention methods are drastically needed in order to provide a more inclusive training environment. This project aims to solve this problem via the use of multimodal signal processing of wearable device signals. When coupled with the Sensor Hub system developed by the industry partner, Thales Canada, the project proposes to quantify cybersickness under four different locomotion conditions and develop a cybersickness index that can be used in trainee evaluations. In particular, locomotion in VR via steering with controllers, teleporting, and physical movement will be compared against physical movement in mixed reality. Cybersickness indices will be developed based on signals measured in real-time from an in-house developed instrumented headset, a smartshirt, and a smartwatch. Anthropometric measures will also be explored in order to remove any potential biases that may be caused by biological sex. The tools developed herein will enable new applications for the industry partner's Sensor Hub, allowing them to reach a new clientele relying on emerging VR/MR applications, as well as provide invaluable insights and tools for Public Safety Canada and the RCMP to make their training and evaluation protocols more inclusive and equitable.