A cloud-based Machine Learning Framework for Assessment of Stress/Engagement through Multimodal Sensors

增加的压力和焦虑会在人体内造成各种问题，特别是对免疫系统有害。在手术前针对性地释放压力可以增加患者更快愈合的能力。非药物压力检测，例如使用电子游戏、音乐和基于智能手机的行为干预来分散注意力，可以带来积极的结果，而不会带来相关的风险。增强/虚拟现实被发现在有压力的临床环境中有特定的好处，包括改善情绪健康，增加积极情绪变化和减少负面情绪。Shaftesbury Inc.与医院和诊所合作，一直在开发一种积极的分心娱乐系统(PDES)。该系统的目的是基于生理信号(例如心率)自动检测用户中的压力水平，并提供适应性体验，其中该体验(例如电影/游戏)基于用户中评估的压力水平而被动态调整。这样的系统可以超越医疗保健领域，通过动态体验提供高娱乐价值。然而，无论在哪个领域，PDES系统中的核心挑战是开发一种可靠的算法来评估用户在体验娱乐产品时的压力/参与度。仅仅利用单一的生理信号，如心率，将不足以准确地评估压力/投入程度。这个项目的目的是开发一个基于云的多模式机器学习框架，用于根据多种生理信号(如心率、肌电、呼吸)和行为信号(如手势、面部表情)为用户自动评估压力/参与度水平，并提供直观和可解释的可视化和分析工具，使非技术利益攸关方能够轻松微调此类算法。拟议的研究将有助于使加拿大在医疗保健和娱乐领域采用XR和机器学习等先进技术方面处于领先地位，而由此产生的对加拿大工业的技术转移将增强加拿大的全球竞争力，并对加拿大经济和社会产生积极影响。

Increased stress and anxiety can create a variety of problems within the human body, and can be especially detrimental to the immune system. Targeting stress relief prior to surgical operations can increase the ability of patients to heal faster. Non-pharmacological stress detection such as distraction using video games, music, and smartphone based behavioural interventions can deliver positive results without associated risks. Augmented/ Virtual Reality has been found to have specific benefits when used in stressful clinical settings, including, improved emotional well-being, increased positive mood shifts and decreased negative emotions. Shaftesbury Inc., in partnership with hospitals and clinics has been developing a Positive Distraction Entertainment System (PDES). The purpose of the system is to automatically detect stress level in users based on physiological signals (e.g. heartrate) and provide an adaptive experience, where the experience (e.g. a movie/game) is dynamically adapted based on the assessed stress level in users. Such a system can go beyond the realm of healthcare, providing high entertainment value through dynamic experience. However, regardless of the domain, the core challenge in the PDES system is to develop a solid algorithm to assess the stress/engagement level of users when experiencing an entertainment product. Merely utilizing a single physiological signal such as heartrate will not be enough to accurately assess the stress/engagement level. The purpose of this project is to develop a multimodal cloud-based machine learning framework for automatic stress/engagement level assessment for users from multiple physiological (e.g. heartrate, EMG, respiration) and behavioural signals (e.g. gesture, facial expression), and provide intuitive and interpretable visualization and analytics tools that provides non-technological stakeholders the ability to easily fine-tune such algorithms. The proposed research will help to position Canada as a leader in adopting advanced technologies such as XR and machine learning in healthcare and entertainment, while the resultant technology transfer to Canadian industry will strengthen Canada's global competitiveness and create positive impacts to Canadian economy and society.