Early versus Late Modality Fusion of Deep Wearable Sensor Features for Personalized Prediction of Tomorrow’s Mood, Health, and Stress*
Early versus Late Modality Fusion of Deep Wearable Sensor Features for Personalized Prediction of Tomorrow’s Mood, Health, and Stress*
复制标题
深度可穿戴传感器功能的早期与晚期模态融合，用于个性化预测明天的情绪、健康和压力*
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Akane Sano
中科院分区：
文献类型：
--
作者：
Boning Li;Akane Sano
预测情绪、健康和压力可以对精神疾病发出早期警报。来自可穿戴传感器的多模式数据提供了对人的内部状态的严谨和丰富的洞察。最近，基于深度学习的连续高分辨率传感器数据特征在睡眠检测和抑郁症诊断等几个无处不在的情感计算应用中的表现已经超过了统计特征。受此启发，我们研究了以皮肤电导、皮肤温度和加速度数据的深度表征学习为特征的多模式数据融合策略，以预测大学生(N=239)的自我报告的情绪、健康和压力得分(0-100)。我们对早期融合框架的交叉验证结果显示，对于看不见的用户，早期融合框架的预测精度显著高于晚期融合框架(p<；0.05)。因此，我们的发现提醒人们注意低水平融合生理数据模式的好处，并证实深入学习的特征的预测效果。临床相关性-这建立了通过从多个传感器模式自动提取特征，选择适当的融合方案可以将预测新用户未来幸福的错误减少高达13.2%。
Predicting mood, health, and stress can sound an early alarm against mental illness. Multi-modal data from wearable sensors provide rigorous and rich insights into one’s internal states. Recently, deep learning-based features on continuous high-resolution sensor data have outperformed statistical features in several ubiquitous and affective computing applications including sleep detection and depression diagnosis. Motivated by this, we investigate multi-modal data fusion strategies featuring deep representation learning of skin conductance, skin temperature, and acceleration data to predict self-reported mood, health, and stress scores (0 - 100) of college students (N = 239). Our cross-validated results from the early fusion framework exhibit a significantly higher (p < 0.05) prediction precision over the late fusion for unseen users. Therefore, our findings call attention to the benefits of fusing physiological data modalities at a low level and corroborate the predictive efficacy of the deeply learned features.Clinical relevance— This establishes that with automatically extracted features from multiple sensor modalities, choosing the proper scheme of fusion can reduce the errors of predicting new users’ future wellbeing by as much as 13.2%.
DOI：
10.1109/tpami.2018.2798607
发表时间：
2019-02-01
影响因子：
23.6
作者：
Baltrusaitis, Tadas;Ahuja, Chaitanya;Morency, Louis-Philippe
通讯作者：
Morency, Louis-Philippe