Sequential Classification of ASL Signs in the Context of Daily Living Using RF Sensing
Sequential Classification of ASL Signs in the Context of Daily Living Using RF Sensing
复制标题
DOI：
10.1109/radarconf2147009.2021.9455178
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Emre Kurtoğlu;A. Gurbuz;E. Malaia;Darrin J. Griffin;Chris S. Crawford;S. Gurbuz
中科院分区：
文献类型：
--
作者：
Emre Kurtoğlu;A. Gurbuz;E. Malaia;Darrin J. Griffin;Chris S. Crawford;S. Gurbuz
随着小封装、高频雷达系统和强大的机器学习工具的发展，基于RF感测的人类活动和手势识别（HGR）方法已经获得了极大的普及。然而，文献中的大多数HGR实验都是在单独的手势上进行的，并且与之前和之后的运动相隔离。本文研究了在日常生活背景下的美国手语识别问题，它涉及到连续的手语流与日常活动的混合的顺序分类。特别是，本文研究了不同的RF输入表示和融合技术的ASL和触发手势识别任务在日常生活中的情况下，这可能是潜在的手语敏感的人机界面（HCI）的功效。所提出的方法涉及首先检测和分割运动周期，然后通过特征级融合的距离多普勒地图，微多普勒频谱图，和包络分类与双向长短期记忆（BiL-STM）递归神经网络。结果表明，在识别的6个活动和4 ASL的迹象，以及触发标志的检测率为0.93，准确率为93.3%。
RF sensing based human activity and hand gesture recognition (HGR) methods have gained enormous popularity with the development of small package, high frequency radar systems and powerful machine learning tools. However, most HGR experiments in the literature have been conducted on individual gestures and in isolation from preceding and subsequent motions. This paper considers the problem of American sign language (ASL) recognition in the context of daily living, which involves sequential classification of a continuous stream of signing mixed with daily activities. In particular, this paper investigates the efficacy of different RF input representations and fusion techniques for ASL and trigger gesture recognition tasks in a daily living scenario, which can be potentially used for sign language sensitive human-computer interfaces (HCI). The proposed approach involves first detecting and segmenting periods of motion, followed by feature level fusion of the range-Doppler map, micro-Doppler spectrogram, and envelope for classification with a bi-directional long short-term memory (BiL-STM) recurrent neural network. Results show 93.3% accuracy in identification of 6 activities and 4 ASL signs, as well as a trigger sign detection rate of 0.93.