Learning Multimodal Representations for Drowsiness Detection
Learning Multimodal Representations for Drowsiness Detection
复制标题
学习困倦检测的多模态表征
DOI：
10.1109/tits.2021.3105326
复制
发表时间：
2022-08
影响因子：
8.5
通讯作者：
Kun Qian;Tomoya Koike;Toru Nakamura;B. Schuller;Yoshiharu Yamamoto
中科院分区：
文献类型：
--
作者：
Kun Qian;Tomoya Koike;Toru Nakamura;B. Schuller;Yoshiharu Yamamoto
困倦检测是安全驾驶的关键一步。已经在使用普适传感器数据（例如，视频、生理学），以构建自动困倦检测系统。然而，大多数现有方法都是基于复杂的可穿戴设备（例如，脑电图）或计算机视觉算法（例如，眼睛状态分析），这使得相关系统在野外几乎不适用。此外，由于有限的模拟实验，基于这些方法的数据本质上是不足的。有鉴于此，我们提出了一种新颖且易于实现的方法，该方法基于完全非侵入性的多模态机器学习分析，用于驾驶员困倦检测任务。嗜睡程度通过预先设计的方案中的自我报告问卷来估计。首先，我们考虑涉及环境数据（例如，温度、湿度、照度等），其可以被视为经由加速度计或活动记录仪记录的人类活动数据的补充信息。其次，我们证明了由日常生活数据训练的模型仍然可以有效地对模拟器中的受试者进行预测，这可能有利于未来的数据收集方法。最后，我们对不同的机器学习方法进行了全面的研究，包括经典的“浅层”模型和最近的深层模型。实验结果表明，我们提出的方法可以达到64.6%的未加权平均召回率的困倦检测在主体无关的情况下。
Drowsiness detection is a crucial step for safe driving. A plethora of efforts has been invested on using pervasive sensor data (e.g., video, physiology) empowered by machine learning to build an automatic drowsiness detection system. Nevertheless, most of the existing methods are based on complicated wearables (e.g., electroencephalogram) or computer vision algorithms (e.g., eye state analysis), which makes the relevant systems hardly applicable in the wild. Furthermore, data based on these methods are insufficient in nature due to limited simulation experiments. In this light, we propose a novel and easily implemented method based on full non-invasive multimodal machine learning analysis for the driver drowsiness detection task. The drowsiness level was estimated by self-reported questionnaire in pre-designed protocols. First, we consider involving environmental data (e.g., temperature, humidity, illuminance, and further more), which can be regarded as complementary information for the human activity data recorded via accelerometers or actigraphs. Second, we demonstrate that the models trained by daily life data can still be efficient to make predictions for the subject performing in a simulator, which may benefit the future data collection methods. Finally, we make a comprehensive study on investigating different machine learning methods including classic ‘shallow’ models and recent deep models. Experimental results show that, our proposed methods can reach 64.6% unweighted average recall for drowsiness detection in a subject-independent scenario.