Development of TARS Mobile App with Deep Fingertip Detector for the Visually Impaired
Development of TARS Mobile App with Deep Fingertip Detector for the Visually Impaired
复制标题
为视障人士开发带有深指尖探测器的TARS移动应用程序
DOI：
10.1007/978-3-030-58796-3_51
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Yoshihiro Hashimoto
中科院分区：
文献类型：
--
作者：
Yoichi Hosokawa;Tetsushi Miwa;Yoshihiro Hashimoto
我们建议使用带有摄像头和深度学习指尖检测器的智能手机的 TARS 移动应用程序，比使用 PC 或触摸屏更容易实施。该应用程序旨在识别用户使用后置摄像头触摸图像的手，并提供语音指导以及食指触摸作为触发器的图像信息。当用食指或拇指执行手势时，应用程序能够立即检测并输出指尖点，并且它可以有效地触发阅读。假设拇指手势在横向方向上的检测方差减少了 68%，因为与食指手势相比，拇指手势很少移动其他四个手指。通过在应用程序中进行多次检测并输出中值，检测方差可以在横向上减少到73%，在纵向上减少到70%，这表明了多次检测的有效性。这些技术可以有效减少指尖检测的方差。我们还确认，如果设备的倾斜度在 -3.4 毫米到 4 毫米之间，当前应用程序可以识别 12 毫米的差异，横向和纵向的平均准确度为 85.5%。最后，我们开发了 TARS 移动应用程序的基本模型，通过使用智能手机摄像头而不是 PC 或触摸屏，可以更轻松地安装和更便携。
We propose TARS mobile applications that uses a smartphone with a camera and deep learning fingertip detector for easier implementation than using a PC or a touch panel. The app was designed to recognize the user’s hand touching the images with the rear camera and provide voice guidance with the information on the images that the index finger is touching as a trigger. When performing gestures with either the index finger or thumb, the app was able to detect and output the fingertip point without delay, and it was effective as a trigger for reading. Thumb gestures are assumed to have reduced detection variances of 68% in the lateral direction because they rarely move the other four fingers compared to index finger gestures. By performing multiple detections in the application and outputting the median, the variances of detection can be reduced to 73% in the lateral direction and 70% in the longitudinal direction, which shows the effectiveness of multiple detections. These techniques are effective in reducing the variance of fingertip detection. We also confirmed that if the tilt of the device is between −3.4 mm and 4 mm, the current app could identify a 12 mm difference with an accuracy of 85.5% as an average in both of the lateral and longitudinal directions. Finally, we developed a basic model of TARS mobile app that allows easier installation and more portability by using a smart phone camera rather than a PC or a touch panel.
盲校教师语音讲解触觉图形创作工具
DOI：
--
发表时间：
2012
期刊：
--
影响因子：
--
作者：
Yamamoto Kensuke;Izumi Kosaku;Inaba Junki;Takayoshi Daisuke;Yoshie Kotaro;Higuchi Norio
通讯作者：
Higuchi Norio