TARS Mobile App with Deep Fingertip Detector for the Visually Impaired
TARS Mobile App with Deep Fingertip Detector for the Visually Impaired
复制标题
TARS 移动应用程序，配有深指尖探测器，适合视障人士
DOI：
10.1007/978-3-030-39512-4_48
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Giuseppe Lisi
中科院分区：
文献类型：
--
作者：
Tetsushi Miwa;Yoichi Hosokawa;Yoshihiro Hashimoto;Giuseppe Lisi
触觉图形与音频响应系统（TARS）允许视障人士通过合成触觉图形的音频描述来学习地图，图表和数字。目前的TARS是笨重的，因为它依赖于计算机和触摸板来检测指尖坐标。在这里，我们通过实现一个TARS移动的应用程序，赋予基于相机的深指尖检测器，提高便携性。我们实现了Simon等人在2017年提出的手部关键点检测器，并将注意力限制在指尖上。在不同的触觉图形类型、参与者和实验条件（即暗/亮环境、部分遮挡）下进行性能基准测试。在正常情况下，准确度平均为98%，但在部分遮挡手的情况下，拇指的检测准确度降低。这些结果表明，所提出的移动的TARS应用程序的性能与标准TARS相似，同时更便携和直观。
Tactile graphics with an Audio Response System (TARS) allows visually impaired people to learn maps, diagrams, and figures by synthesizing audio descriptions of tactile graphics. The current TARS is cumbersome since it relies on a computer and touch panel to detect fingertip coordinates. Here, we improve portability by implementing a TARS mobile app endowed with a camera-based deep fingertip detector. We implemented the hand keypoint detector proposed inSimon et al. 2017, and restricted our attention on the fingertips. Performance benchmarking was done over different tactile graphics types, participants and experimental conditions (i.e. dark/light environment, partial occlusion). In normal conditions, accuracy is 98% on average, but in the case of a partially occluded hand, the detection accuracy of the thumb decreases. These results indicate that the proposed mobile TARS app would perform similar to the standard TARS while being more portable and intuitive.