Application Informed Motion Signal Processing for Finger Motion Tracking Using Wearable Sensors
Application Informed Motion Signal Processing for Finger Motion Tracking Using Wearable Sensors
复制标题
DOI：
10.1109/icassp40776.2020.9053466
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Yilin Liu;Fengyang Jiang;Mahanth K. Gowda
中科院分区：
文献类型：
--
作者：
Yilin Liu;Fengyang Jiang;Mahanth K. Gowda
手指运动跟踪在用户界面、运动分析、医疗康复和手语翻译方面有许多应用。本文提出了一个名为 FinGTrAC 的系统，该系统展示了使用低侵入性可穿戴传感器平台（戴在食指上的智能戒指和戴在手腕上的智能手表）进行细粒度手指手势跟踪的可行性。这种稀疏的传感器佩戴起来很方便，但无法跟踪所有手指，因此提供的信息受到约束。然而，特定于应用程序的上下文可以填补稀疏感知的空白并提高手势分类的准确性。本文展示了在美国手语 (ASL) 翻译应用中利用此类上下文的可行性。由于传感器数据嘈杂、用户手势性能的差异以及无法从所有手指捕获数据，出现了不小的挑战。 FinGTrAC 利用数据预处理、过滤、模式匹配、ASL 句子上下文等方面的大量机会，将可用的感官信息系统地融合到贝叶斯过滤框架中。最终设计出隐马尔可夫模型，并设计了维特比解码方案来实时检测手指手势和相应的 ASL 句子。对 10 位用户的广泛评估显示，对于不同句子中 100 个最常用的 ASL 手指手势，检测准确率为 94.2%。
Finger motion tracking has a number of applications in user-interfaces, sports analytics, medical rehabilitation and sign language translation. This paper presents a system called FinGTrAC that shows the feasibility of fine grained finger gesture tracking using low intrusive wearable sensor platform (smart-ring worn on the index finger and a smart-watch worn on the wrist). Such sparse sensors are convenient to wear but cannot track all fingers and hence provide under-constrained information. However application specific context can fill the gap in sparse sensing and improve the accuracy of gesture classification. This paper shows the feasibility of exploiting such context in an application of American Sign Language (ASL) translation. Non-trivial challenges arise due to noisy sensor data, variations in gesture performance across users and the inability to capture data from all fingers. FinGTrAC exploits a number of opportunities in data preprocessing, filtering, pattern matching, context of an ASL sentence to systematically fuse the available sensory information into a Bayesian filtering framework. Culminating into the design of a Hidden Markov Model, a Viterbi decoding scheme is designed to detect finger gestures and the corresponding ASL sentences in real time. Extensive evaluation on 10 users shows a detection accuracy of 94.2% for 100 most frequently used ASL finger gestures over different sentences.