Multimodal Recognition of Reading Activity in Transit Using Body-Worn Sensors
Multimodal Recognition of Reading Activity in Transit Using Body-Worn Sensors
复制标题
DOI：
10.1145/2134203.2134205
复制
发表时间：
2012-03-01
影响因子：
1.6
通讯作者：
Gellersen, Hans
中科院分区：
文献类型：
--
作者：
Bulling, Andreas;Ward, Jamie A.;Gellersen, Hans
阅读是研究最深入的视觉活动之一。视觉研究传统上侧重于理解阅读过程中的知觉和认知过程。在这项工作中，我们通过联合分析人们在日常环境中的眼睛和头部运动来识别阅读活动。眼球运动使用眼电图仪(EOG)系统记录；身体运动使用身体佩戴的惯性测量单元。我们比较了两种连续识别阅读的方法：字符串匹配法(STR)，它显式地建模阅读过程中的特征水平扫视，以及支持向量机(SVM)，它依赖于从眼动数据中提取的90个眼动特征。我们在一项研究中对这两种方法进行了评估，研究对象包括八名参与者，他们坐在办公桌前、站着、在室内和室外散步以及乘坐有轨电车阅读。我们介绍了一种利用阅读过程中眼睛和头部运动的感觉运动协调来分割阅读活动的方法。使用独立于人的训练，我们获得了88.9%的识别率(召回率为72.3%)和87.7%的识别率(召回率为87.9%)。实验结果表明，该分割方法将阅读事件的识别性能提高了24%以上。我们的工作表明，对眼睛和身体运动的联合分析有利于阅读识别，并开启了关于多模式识别方法在其他视觉和身体活动中的更广泛适用性的讨论。
Reading is one of the most well-studied visual activities. Vision research traditionally focuses on understanding the perceptual and cognitive processes involved in reading. In this work we recognize reading activity by jointly analyzing eye and head movements of people in an everyday environment. Eye movements are recorded using an electrooculography (EOG) system; body movements using body-worn inertial measurement units. We compare two approaches for continuous recognition of reading: String matching (STR) that explicitly models the characteristic horizontal saccades during reading, and a support vector machine (SVM) that relies on 90 eye movement features extracted from the eye movement data. We evaluate both methods in a study performed with eight participants reading while sitting at a desk, standing, walking indoors and outdoors, and riding a tram. We introduce a method to segment reading activity by exploiting the sensorimotor coordination of eye and head movements during reading. Using person-independent training, we obtain an average precision for recognizing reading of 88.9% (recall 72.3%) using STR and of 87.7% (recall 87.9%) using SVM over all participants. We show that the proposed segmentation scheme improves the performance of recognizing reading events by more than 24%. Our work demonstrates that the joint analysis of eye and body movements is beneficial for reading recognition and opens up discussion on the wider applicability of a multimodal recognition approach to other visual and physical activities.