Automatic Detection of Mind Wandering from Video in the Lab and in the Classroom
Automatic Detection of Mind Wandering from Video in the Lab and in the Classroom
复制标题
DOI：
10.1109/taffc.2019.2908837
复制
发表时间：
2021-10-01
影响因子：
11.2
通讯作者：
D'Mello, Sidney K.
中科院分区：
文献类型：
--
作者：
Bosch, Nigel;D'Mello, Sidney K.
我们报告了两项使用面部特征自动检测走神的研究，走神是一种普遍存在的现象，即注意力从当前任务转移到无关的想法。在实验室研究中，大学生（N = 152）阅读科学文本，而在课堂研究中，高中生（N = 135）从智能辅导系统中学习生物学。使用有效的自我报告方法测量走神。在实验室中，我们记录了面部视频，并在六个粒度级别上进行了分析：（1）上半身运动;（2）头部姿势;（3）面部纹理;（4）面部动作单元（AU）;（5）共同出现的AU;（6）AU的时间动态。由于隐私限制，视频没有在教室里录制。相反，我们实时提取头部姿势、AU和Au同现。由支持向量机（SVM）和深度神经网络组成的机器学习模型在实验室和教室中分别检测到了.478和.414的F-1分数（25.4%和20.9%的机会改进，均使用SVM）。基于实验室的探测器比以前的最先进的探测器提高了8.4%;教室探测器没有可比性。我们讨论了如何将探测器集成到智能界面中，以通过响应走神来增加参与和学习。
We report two studies that used facial features to automatically detect mind wandering, a ubiquitous phenomenon whereby attention drifts from the current task to unrelated thoughts. In a laboratory study, university students (N = 152) read a scientific text, whereas in a classroom study high school students (N = 135) learned biology from an intelligent tutoring system. Mind wandering was measured using validated self-report methods. In the lab, we recorded face videos and analyzed these at six levels of granularity: (1) upper-body movement; (2) head pose; (3) facial textures; (4) facial action units (AUs); (5) co-occurring AUs; and (6) temporal dynamics of AUs. Due to privacy constraints, videos were not recorded in the classroom. Instead, we extracted head pose, AUs, and AU co-occurrences in real-time. Machine learning models, consisting of support vector machines (SVM) and deep neural networks, achieved F-1 scores of.478 and .414 (25.4 and 20.9 percent above-chance improvements, both with SVMs) for detecting mind wandering in the lab and classroom, respectively. The lab-based detectors achieved 8.4 percent improvement over the previous state-of-the-art; no comparison is available for classroom detectors. We discuss how the detectors can integrate into intelligent interfaces to increase engagement and learning by responding to wandering minds.