Investigating the feasibility of finger identification on capacitive touchscreens using deep learning
Investigating the feasibility of finger identification on capacitive touchscreens using deep learning
复制标题
使用深度学习研究电容式触摸屏上手指识别的可行性
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
N. Henze
中科院分区：
文献类型：
--
作者：
Huy Viet Le;Sven Mayer;N. Henze
触摸屏可实现直观的移动交互。然而，触摸输入仅限于 2D 触摸位置，这使得提供类似于硬件键盘和鼠标的快捷方式和辅助操作变得具有挑战性。之前的工作提出了多种方法，通过识别哪个手指触摸了显示屏来提供辅助操作。虽然这些方法基于不方便的外部传感器，但我们使用移动触摸屏的电容图像来研究手指识别的可行性。我们收集了低分辨率指纹数据集，并训练了卷积神经网络，对八种手指组合的触摸进行分类。我们专注于涉及拇指和食指的组合，因为它们主要用于交互。结果，我们实现了左右拇指位置不变区分的准确率超过 92%。我们评估了该模型和两个用户认为有用且直观的用例。我们公开分享我们的数据集 (CapFingerld)，其中包含 455,709 个具有代表性的互电容式触摸屏上每个手指触摸的电容图像以及我们的模型，以便将来的工作能够使用和改进它们。
Touchscreens enable intuitive mobile interaction. However, touch input is limited to 2D touch locations which makes it challenging to provide shortcuts and secondary actions similar to hardware keyboards and mice. Previous work presented a wide range of approaches to provide secondary actions by identifying which finger touched the display. While these approaches are based on external sensors which are inconvenient, we use capacitive images from mobile touchscreens to investigate the feasibility of finger identification. We collected a dataset of low-resolution fingerprints and trained convolutional neural networks that classify touches from eight combinations of fingers. We focused on combinations that involve the thumb and index finger as these are mainly used for interaction. As a result, we achieved an accuracy of over 92% for a position-invariant differentiation between left and right thumbs. We evaluated the model and two use cases that users find useful and intuitive. We publicly share our data set (CapFingerld) comprising 455,709 capacitive images of touches from each finger on a representative mutual capacitive touchscreen and our models to enable future work using and improving them.