FaceSense: Sensing Face Touch with an Ear-worn System
FaceSense: Sensing Face Touch with an Ear-worn System
复制标题
DOI：
10.1145/3478129
复制
发表时间：
2021-09-01
期刊：
影响因子：
--
通讯作者：
Vu, Tam
中科院分区：
文献类型：
--
作者：
Kakaraparthi, Vimal;Shao, Qijia;Vu, Tam
摸脸是人类无意识的习惯。频繁触摸面部敏感/粘膜区域（眼睛、鼻子和嘴）会将病原体传入体内并传播疾病，从而增加健康风险。此外，准确监测面部触摸对于行为干预至关重要。现有的监控系统仅捕获接近面部的物体，而不是检测实际触摸。因此，这些系统在接近人的面部的手或物体移动时容易出现误报（例如，拿起电话）。我们提出了FaceSense，一种耳戴式系统，能够识别实际的触摸，并将它们与其他面部区域的敏感/粘膜区域区分开来。FaceSense采用多模式方法，集成了低分辨率热图像和生理信号。热传感器感测由接近的手发出的热红外信号，而生理传感器监测触摸期间皮肤变形引起的阻抗变化。处理后的热信号和生理信号被输入深度学习模型（TouchNet），以检测触摸并识别触摸的面部区域。我们使用现成的硬件制造了原型，并在14名参与者进行各种日常活动（例如，喝酒，聊天）。结果显示，使用留一用户交叉验证的触摸检测的宏F1分数为83.4%，使用个性化模型的触摸区域识别的宏F1分数为90.1%。
Face touch is an unconscious human habit. Frequent touching of sensitive/mucosal facial zones (eyes, nose, and mouth) increases health risks by passing pathogens into the body and spreading diseases. Furthermore, accurate monitoring of face touch is critical for behavioral intervention. Existing monitoring systems only capture objects approaching the face, rather than detecting actual touches. As such, these systems are prone to false positives upon hand or object movement in proximity to one's face (e.g., picking up a phone). We present FaceSense, an ear-worn system capable of identifying actual touches and differentiating them between sensitive/mucosal areas from other facial areas. Following a multimodal approach, FaceSense integrates low-resolution thermal images and physiological signals. Thermal sensors sense the thermal infrared signal emitted by an approaching hand, while physiological sensors monitor impedance changes caused by skin deformation during a touch. Processed thermal and physiological signals are fed into a deep learning model (TouchNet) to detect touches and identify the facial zone of the touch. We fabricated prototypes using off-the-shelf hardware and conducted experiments with 14 participants while they perform various daily activities (e.g., drinking, talking). Results show a macro-F1-score of 83.4% for touch detection with leave-one-user-out cross-validation and a macro-F1-score of 90.1% for touch zone identification with a personalized model.