Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study
Sensor-Based Prediction of Mental Effort during Learning from Physiological Data: A Longitudinal Case Study
复制标题
DOI：
10.3390/signals2040051
复制
发表时间：
2021-12-01
期刊：
影响因子：
--
通讯作者：
Romine, William
中科院分区：
文献类型：
--
作者：
Agarwal, Ankita;Graft, Josephine;Romine, William
活动和身体健康的跟踪器已经变得无处不在。虽然最近的工作已经证明了显着的心理努力和生理数据，如皮肤温度，心率和皮肤电活动之间的关系，我们还没有证明他们的有效性预测的心理努力，这样一个有用的心理努力跟踪器可以开发。考虑到先前难以提取个体之间可重复的心理努力和生理反应之间的关系，我们认为，将自我报告措施与互联网或智能手机应用程序中的生理数据融合可能会提供一种有效的方法来训练有用的心理努力跟踪系统。在这个案例研究中，我们利用了一个大学学期的课程中来自单个参与者的90多个小时的数据。通过将参与者在学期中不同活动中的自我报告的脑力劳动与Empatica E4可穿戴传感器收集的并发生理数据相融合，我们探讨了训练这样一个设备需要多少数据，以及哪种类型的机器学习算法最有效。我们得出的结论是，尽管逻辑回归和马尔可夫模型等基线模型提供了关于学生的生理如何随着心理努力而变化的有用解释信息，但深度学习算法能够使用前28小时的数据进行训练来生成准确的预测。建议使用一个结合了长短期记忆和卷积神经网络的系统，以便生成平滑的预测，同时还能够捕捉使用该设备的个人在精神努力方面的转变。
Trackers for activity and physical fitness have become ubiquitous. Although recent work has demonstrated significant relationships between mental effort and physiological data such as skin temperature, heart rate, and electrodermal activity, we have yet to demonstrate their efficacy for the forecasting of mental effort such that a useful mental effort tracker can be developed. Given prior difficulty in extracting relationships between mental effort and physiological responses that are repeatable across individuals, we make the case that fusing self-report measures with physiological data within an internet or smartphone application may provide an effective method for training a useful mental effort tracking system. In this case study, we utilized over 90 h of data from a single participant over the course of a college semester. By fusing the participant's self-reported mental effort in different activities over the course of the semester with concurrent physiological data collected with the Empatica E4 wearable sensor, we explored questions around how much data were needed to train such a device, and which types of machine-learning algorithms worked best. We concluded that although baseline models such as logistic regression and Markov models provided useful explanatory information on how the student's physiology changed with mental effort, deep-learning algorithms were able to generate accurate predictions using the first 28 h of data for training. A system that combines long short-term memory and convolutional neural networks is recommended in order to generate smooth predictions while also being able to capture transitions in mental effort when they occur in the individual using the device.