Stress assessment using multimodal learning in an augmented reality environment

点击翻译按钮获取中文摘要

Anxiety or stress before a complex pediatric procedure is detrimental to a child's health. It also adds cost to the**healthcare system by potentially prolonging the recovery period. Shaftesbury Development II Inc is developing**a a Positive Distraction Entertainment System (PDES) that can dynamically deliver game/media content to**alleviate the user's stress for maximum positive outcome. It can be used as a mean for non-pharmacological**anxiety reduction in children before a complex pediatric procedure. The PDES system dynamically adjusts the**media content (e.g. an interactive Virtual Reality (VR) game) based on the user's current stress level which can**be assessed through biomedical signals such as heart-rate, EEG. Currently, Shaftesbury is investigating how to**extend the current system into Augmented Reality (AR) headsets such as Microsoft Hololens, which will**provide the opportunity to develop collaborative experiences with multiple users. The camera input on such**devices also opens up the opportunity to incorporate behavioral signals (e.g. user's hand gesture, facial**expression) alongside biomedical signals to further strengthen the stress level and engagement assessment. In**this research project, we will employ state-of-the-art machine learning techniques to fuse multimodal**(behavioral and biomedical) signals to accurately assess the stress level of a user. Our proposed system will**depend on very little training data due to implementation of novel signal transformation and transfer learning**techniques. The system will be fully integrated with Shaftesbury's PDES so that game/media content can be**adjusted dynamically to reduce stress before a medical procedure.