BoMW: Bag of Manifold Words for One-Shot Learning Gesture Recognition From Kinect
BoMW: Bag of Manifold Words for One-Shot Learning Gesture Recognition From Kinect
复制标题
BoMW：用于从 Kinect 一次性学习手势识别的流形词袋
DOI：
10.1109/tcsvt.2017.2721108
复制
发表时间：
2018-10-01
影响因子：
8.4
通讯作者：
Zhou, Huiyu
中科院分区：
文献类型：
--
作者：
Zhang, Lei;Zhang, Shengping;Zhou, Huiyu
在本文中，我们研究了一次性学习手势识别从微软的Kinect记录的RGB-D数据。为此，我们提出了一种新的袋流形词（BoMW）的对称正定（SPD）流形上的特征表示。特别是，我们使用协方差矩阵从RGB-D数据中提取局部特征，因为它具有紧凑的表示能力以及融合RGB和深度信息的方便性。由于协方差矩阵是SPD矩阵，并且它们所跨越的空间是SPD流形，因此传统的欧几里得空间中的学习方法，例如稀疏编码，不能直接应用于它们。为了克服这个问题，我们提出了一个统一的框架，将SPD流形上的稀疏编码转移到欧氏空间上，这使得任何现有的学习方法都可以使用。在从每个手势类的视频上构建BoMW表示之后，采用最近邻分类器来执行一次性学习手势识别。在ChaLearn手势数据集上的实验结果表明，本文提出的单次学习手势识别方法与现有的方法相比具有优异的性能。在一个新的RGB-D动作识别数据集上验证了所提特征提取方法的有效性。
In this paper, we study one-shot learning gesture recognition on RGB-D data recorded from Microsoft's Kinect. To this end, we propose a novel bag of manifold words (BoMW)based feature representation on symmetric positive definite (SPD) manifolds. In particular, we use covariance matrices to extract local features from RGB-D data due to its compact representation ability as well as the convenience of fusing both RGB and depth information. Since covariance matrices are SPD matrices and the space spanned by them is the SPD manifold, traditional learning methods in the Euclidean space, such as sparse coding, cannot be directly applied to them. To overcome this problem, we propose a unified framework to transfer the sparse coding on SPD manifolds to the one on the Euclidean space, which enables any existing learning method to be used. After building BoMW representation on a video from each gesture class, a nearest neighbor classifier is adopted to perform the one-shot learning gesture recognition. Experimental results on the ChaLearn gesture data set demonstrate the outstanding performance of the proposed one-shot learning gesture recognition method compared against the state-of-the-art methods. The effectiveness of the proposed feature extraction method is also validated on a new RGB-D action recognition data set.