Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study
Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study
复制标题
使用智能手表传感器数据进行情绪识别：混合设计研究
作者：
Juan C. Quiroz;E. Geangu;M. Yong
心理学研究表明，一个人走路的方式反映了这个人当前的情绪（或情绪状态）。最近的研究已经使用移动的电话从运动数据中检测情绪状态。目的本研究的目的是调查使用智能手表的运动传感器数据来推断个人的情绪状态。我们提出了我们的用户研究结果与50名参与者。方法实验设计为混合设计研究：被试内（情绪：快乐、悲伤和中性）和被试间（刺激类型：视听“电影片段”和音频“音乐片段”）。每个参与者都经历了单一刺激类型的两种情绪。所有参与者步行250米，同时在一个手腕上佩戴智能手表，并在胸部佩戴心率监测器。他们还必须在经历每种情绪之前和之后回答一份简短的问卷（20个项目;积极情绪和消极情绪量表，PANAS）。从心率监测器获得的数据作为我们数据的补充信息。我们对智能手表的数据进行了时间序列分析，并对问卷项目进行了t检验，以衡量情绪状态的变化。心率数据采用单因素方差分析进行分析。我们使用滑动窗口从时间序列中提取特征，并使用这些特征来训练和验证确定个人情绪的分类器。结果共有50名年轻人参与了我们的研究，其中49人被纳入情感PANAS问卷，44人被纳入特征提取和个人模型的建立。参与者报告说，在观看悲伤视频或听悲伤音乐后，他们的负面情绪减少了，P<0.006。对于使用分类器进行情感识别的任务，我们的结果表明，个人模型的表现优于个人基线，并且在快乐与悲伤的二元分类设计研究的所有条件下，平均准确率均高于78%。结论我们的研究结果表明，我们能够利用从智能手表获得的数据检测情绪状态和行为反应的变化。连同所有用户在快乐与悲伤情绪状态的分类中实现的高准确度，这进一步证明了运动传感器数据可用于情绪识别的假设。
Background Research in psychology has shown that the way a person walks reflects that person’s current mood (or emotional state). Recent studies have used mobile phones to detect emotional states from movement data. Objective The objective of our study was to investigate the use of movement sensor data from a smart watch to infer an individual’s emotional state. We present our findings of a user study with 50 participants. Methods The experimental design is a mixed-design study: within-subjects (emotions: happy, sad, and neutral) and between-subjects (stimulus type: audiovisual “movie clips” and audio “music clips”). Each participant experienced both emotions in a single stimulus type. All participants walked 250 m while wearing a smart watch on one wrist and a heart rate monitor strap on the chest. They also had to answer a short questionnaire (20 items; Positive Affect and Negative Affect Schedule, PANAS) before and after experiencing each emotion. The data obtained from the heart rate monitor served as supplementary information to our data. We performed time series analysis on data from the smart watch and a t test on questionnaire items to measure the change in emotional state. Heart rate data was analyzed using one-way analysis of variance. We extracted features from the time series using sliding windows and used features to train and validate classifiers that determined an individual’s emotion. Results Overall, 50 young adults participated in our study; of them, 49 were included for the affective PANAS questionnaire and 44 for the feature extraction and building of personal models. Participants reported feeling less negative affect after watching sad videos or after listening to sad music, P<.006. For the task of emotion recognition using classifiers, our results showed that personal models outperformed personal baselines and achieved median accuracies higher than 78% for all conditions of the design study for binary classification of happiness versus sadness. Conclusions Our findings show that we are able to detect changes in the emotional state as well as in behavioral responses with data obtained from the smartwatch. Together with high accuracies achieved across all users for classification of happy versus sad emotional states, this is further evidence for the hypothesis that movement sensor data can be used for emotion recognition.