A cloud-based Machine Learning Framework for Assessment of Stress/Engagement through Multimodal Sensors

压力和焦虑的增加会在人体内产生各种问题，并且可能对免疫系统特别有害。在手术前进行有针对性的压力缓解可以提高患者更快愈合的能力。非药物压力检测，例如使用视频游戏，音乐和基于智能手机的行为干预来分散注意力，可以提供积极的结果，而不会产生相关风险。研究发现，增强/虚拟现实在压力较大的临床环境中使用时具有特定的好处，包括改善情绪健康、增加积极情绪转变和减少消极情绪。沙夫茨伯里公司，与医院和诊所合作，开发了一个积极的娱乐系统（PDES）。该系统的目的是基于生理信号（例如，心率）自动检测用户的压力水平，并提供自适应体验，其中基于用户的评估压力水平动态地调整体验（例如，电影/游戏）。这样的系统可以超越医疗保健领域，通过动态体验提供高娱乐价值。然而，无论领域如何，PDES系统的核心挑战是开发一种可靠的算法来评估用户在体验娱乐产品时的压力/参与程度。仅仅利用诸如心率的单个生理信号将不足以准确地评估压力/参与水平。 该项目的目的是开发一个基于云的多模态机器学习框架，用于从多个生理（例如心率，EMG，呼吸）和行为信号（例如手势，面部表情）中为用户进行自动压力/参与水平评估，并提供直观和可解释的可视化和分析工具，为非技术利益相关者提供轻松微调此类算法的能力。拟议的研究将有助于将加拿大定位为在医疗保健和娱乐领域采用XR和机器学习等先进技术的领导者，而由此产生的技术转让将加强加拿大的全球竞争力，并对加拿大经济和社会产生积极影响。

Increased stress and anxiety can create a variety of problems within the human body, and can be especially detrimental to the immune system. Targeting stress relief prior to surgical operations can increase the ability of patients to heal faster. Non-pharmacological stress detection such as distraction using video games, music, and smartphone based behavioural interventions can deliver positive results without associated risks. Augmented/ Virtual Reality has been found to have specific benefits when used in stressful clinical settings, including, improved emotional well-being, increased positive mood shifts and decreased negative emotions. Shaftesbury Inc., in partnership with hospitals and clinics has been developing a Positive Distraction Entertainment System (PDES). The purpose of the system is to automatically detect stress level in users based on physiological signals (e.g. heartrate) and provide an adaptive experience, where the experience (e.g. a movie/game) is dynamically adapted based on the assessed stress level in users. Such a system can go beyond the realm of healthcare, providing high entertainment value through dynamic experience. However, regardless of the domain, the core challenge in the PDES system is to develop a solid algorithm to assess the stress/engagement level of users when experiencing an entertainment product. Merely utilizing a single physiological signal such as heartrate will not be enough to accurately assess the stress/engagement level. The purpose of this project is to develop a multimodal cloud-based machine learning framework for automatic stress/engagement level assessment for users from multiple physiological (e.g. heartrate, EMG, respiration) and behavioural signals (e.g. gesture, facial expression), and provide intuitive and interpretable visualization and analytics tools that provides non-technological stakeholders the ability to easily fine-tune such algorithms. The proposed research will help to position Canada as a leader in adopting advanced technologies such as XR and machine learning in healthcare and entertainment, while the resultant technology transfer to Canadian industry will strengthen Canada's global competitiveness and create positive impacts to Canadian economy and society.