Detection and prediction of cybersickness in virtual and mixed reality environments using wearables

虚拟和混合现实（VR/MR）系统在过去几年中蓬勃发展，应用于医疗保健，游戏，远程呈现和技能培训等领域。例如，在技能培训部门，警察/执法培训是一个重要的应用领域，在世界范围内得到越来越多的采用。在加拿大，公共安全部门和皇家加拿大骑警（RCMP）投资数百万美元建立了最先进的VR/MR设施，以培训下一代执法人员。VR允许在一个物理位置测试不同的场景、条件和动作，从而不仅降低了培训成本，而且更好地装备学员处理未知事物。虽然潜力是存在的，但已知现有的VR/MR系统会引起大部分受训者，特别是女性的晕动病-称为晕电病。因此，为了提供一个更具包容性的培训环境，迫切需要晕机检测和预防方法。该项目旨在通过使用可穿戴设备信号的多模态信号处理来解决这个问题。当与行业合作伙伴Thales Canada开发的传感器集线器系统相结合时，该项目建议在四种不同的运动条件下量化晕网病，并开发可用于受训者评估的晕网病指数。特别是，VR中通过控制器转向、心灵传送和物理移动的运动将与混合现实中的物理移动进行比较。电脑病指数将根据从内部开发的仪器化耳机、智能衬衫和智能手表实时测量的信号来制定。还将探讨人体测量措施，以消除可能由生物性别引起的任何潜在偏见。本文开发的工具将为行业合作伙伴的传感器中心提供新的应用程序，使他们能够依靠新兴的VR/MR应用程序接触新的客户，并为加拿大公共安全部和皇家骑警提供宝贵的见解和工具，使他们的培训和评估协议更具包容性和公平性。

Virtual and mixed reality (VR/MR) systems have burgeoned over the last couple of years with applications in healthcare, gaming, telepresence, and skills training, to name a few. Within the skills training sector, for example, police/law enforcement training is an important application domain which has seen increased adoption worldwide. In Canada, Public Safety and the Royal Canadian Mounted Police (RCMP) have invested millions of dollars to set up state-of-the-art VR/MR facilities to train the next generation of law enforcement agents. VR allows for different scenes, conditions, and maneuvers to be tested in one single physical location, thus not only reducing training costs, but better equipping trainees to handle unknowns. While the potential is there, existing VR/MR systems are known to induce motion sickness - known as cybersickness - on a large proportion of the trainees, especially females. As such, cybersickness detection and prevention methods are drastically needed in order to provide a more inclusive training environment. This project aims to solve this problem via the use of multimodal signal processing of wearable device signals. When coupled with the Sensor Hub system developed by the industry partner, Thales Canada, the project proposes to quantify cybersickness under four different locomotion conditions and develop a cybersickness index that can be used in trainee evaluations. In particular, locomotion in VR via steering with controllers, teleporting, and physical movement will be compared against physical movement in mixed reality. Cybersickness indices will be developed based on signals measured in real-time from an in-house developed instrumented headset, a smartshirt, and a smartwatch. Anthropometric measures will also be explored in order to remove any potential biases that may be caused by biological sex. The tools developed herein will enable new applications for the industry partner's Sensor Hub, allowing them to reach a new clientele relying on emerging VR/MR applications, as well as provide invaluable insights and tools for Public Safety Canada and the RCMP to make their training and evaluation protocols more inclusive and equitable.