Robust Part-Based Hand Gesture Recognition Using Kinect Sensor
Robust Part-Based Hand Gesture Recognition Using Kinect Sensor
复制标题
DOI：
10.1109/tmm.2013.2246148
复制
发表时间：
2013-08-01
影响因子：
7.3
通讯作者：
Zhang, Zhengyou
中科院分区：
文献类型：
--
作者：
Ren, Zhou;Yuan, Junsong;Zhang, Zhengyou
最近开发的深度传感器，如Kinect传感器，为人机交互(HCI)提供了新的机会。虽然利用Kinect传感器在人体跟踪、人脸识别和人体动作识别等方面已经取得了很大的进步，但稳健的手势识别仍然是一个悬而未决的问题。与整个人体相比，手是一个更小的物体，关节更复杂，更容易受到分割错误的影响。因此，手势识别是一个非常具有挑战性的问题。本文利用Kinect传感器构建了一个健壮的基于部件的手势识别系统。为了处理从Kinect传感器获得的噪声手形，我们提出了一种新的距离度量，FEMD来度量手形之间的差异性。由于它只匹配手指部分，而不匹配整个手，因此可以更好地区分细微差异的手势。大量的实验表明，我们的手势识别系统是准确的(在具有挑战性的10个手势数据集上的平均准确率为93.2%)，高效(平均每帧0.0750 S)，对手部发音、扭曲和方向或比例变化具有较强的鲁棒性，并且可以在不受控制的环境(杂乱的背景和光照条件)下工作。在两个真实的人机界面应用中，我们的系统的优越性进一步得到了证明。
The recently developed depth sensors, e. g., the Kinect sensor, have provided new opportunities for human-computer interaction (HCI). Although great progress has been made by leveraging the Kinect sensor, e. g., in human body tracking, face recognition and human action recognition, robust hand gesture recognition remains an open problem. Compared to the entire human body, the hand is a smaller object with more complex articulations and more easily affected by segmentation errors. It is thus a very challenging problem to recognize hand gestures. This paper focuses on building a robust part-based hand gesture recognition system using Kinect sensor. To handle the noisy hand shapes obtained from the Kinect sensor, we propose a novel distance metric, Finger-EarthMover's Distance (FEMD), to measure the dissimilarity between hand shapes. As it only matches the finger parts while not the whole hand, it can better distinguish the hand gestures of slight differences. The extensive experiments demonstrate that our hand gesture recognition system is accurate (a 93.2% mean accuracy on a challenging 10-gesture dataset), efficient (average 0.0750 s per frame), robust to hand articulations, distortions and orientation or scale changes, and can work in uncontrolled environments (cluttered backgrounds and lighting conditions). The superiority of our system is further demonstrated in two real-life HCI applications.