fNIRS-based classification of mind-wandering with personalized window selection for multimodal learning interfaces
fNIRS-based classification of mind-wandering with personalized window selection for multimodal learning interfaces
复制标题
DOI：
10.1007/s12193-020-00325-z
复制
发表时间：
2020-06
影响因子：
2.9
通讯作者：
Ruixue Liu;Erin Walker;Leah Friedman;Catherine M. Arrington;E. Solovey
中科院分区：
文献类型：
--
作者：
Ruixue Liu;Erin Walker;Leah Friedman;Catherine M. Arrington;E. Solovey
对个人走神状态的自动检测对设计和评估吸引人的有效学习界面具有重要意义。虽然很难区分一个人是在走神还是只根据外部可观察到的行为专注于任务，但基于大脑的感知提供了对内部状态的独特见解。为了探索可行性，我们使用功能近红外光谱（fNIRS）进行了一项研究，并研究了基于fNIRS数据的机器学习分类器在个人和群体层面检测走神事件，特别关注自动窗口选择以提高分类结果。对于个体水平分类，我们采用移动窗口法结合线性判别分类器，找到了最佳的分类窗口，平均f1得分为74.8%。对于群体水平的分类，我们提出了一种基于个体的时间窗口选择（ITWS）算法，将个体差异纳入窗口选择。该算法首先通过使用嵌入的个人级分类器为每个个体找到最佳窗口，然后使用来自所有参与者的这些窗口构建最终分类器。当与极端梯度增强、卷积神经网络和深度神经网络一起使用时，对ITWS算法的性能进行了评估。我们的研究结果表明，该算法在基于大脑的走神分类方面取得了显著的进步，平均f1得分为73.2%。这为多模态学习界面的评估和未来的注意力感知系统的走神检测奠定了基础。
Automatic detection of an individual’s mind-wandering state has implications for designing and evaluating engaging and effective learning interfaces. While it is difficult to differentiate whether an individual is mind-wandering or focusing on the task only based on externally observable behavior, brain-based sensing offers unique insights to internal states. To explore the feasibility, we conducted a study using functional near-infrared spectroscopy (fNIRS) and investigated machine learning classifiers to detect mind-wandering episodes based on fNIRS data, both on an individual level and a group level, specifically focusing on automated window selection to improve classification results. For individual-level classification, by using a moving window method combined with a linear discriminant classifier, we found the best windows for classification and achieved a mean F1-score of 74.8%. For group-level classification, we proposed an individual-based time window selection (ITWS) algorithm to incorporate individual differences in window selection. The algorithm first finds the best window for each individual by using embedded individual-level classifiers and then uses these windows from all participants to build the final classifier. The performance of the ITWS algorithm is evaluated when used with eXtreme gradient boosting, convolutional neural networks, and deep neural networks. Our results show that the proposed algorithm achieved significant improvement compared to the previous state of the art in terms of brain-based classification of mind-wandering, with an average F1-score of 73.2%. This builds a foundation for mind-wandering detection for both the evaluation of multimodal learning interfaces and for future attention-aware systems.