Robust human activity recognition from depth video using spatiotemporal multi-fused features
Robust human activity recognition from depth video using spatiotemporal multi-fused features
复制标题
DOI：
10.1016/j.patcog.2016.08.003
复制
发表时间：
2017-01-01
影响因子：
8
通讯作者：
Kim, Daijin
中科院分区：
文献类型：
--
作者：
Jalal, Ahmad;Kim, Yeon-Ho;Kim, Daijin
最近开发的深度成像技术为人类活动识别（HAR）提供了新的方向，而无需将光学标记或任何其他运动传感器附着到人体部位。在本文中，我们提出了新的多融合功能的在线人类活动识别（HAR）系统，识别人类活动的连续序列的深度图。提出的在线HAR系统分割人体深度轮廓使用的时间人体运动信息，以及它获得人体骨骼关节使用时空人体信息。然后，它提取的时空多融合功能，串联四个骨架关节特征和一个身体形状特征。骨架关节特征包括基于躯干的距离特征（DT）、基于关键关节的距离特征（DK）、时空幅度特征（M）和时空方向角特征（theta）。被称为HOG-DDS的身体形状特征通过方向梯度直方图（HOG）格式表示两个连续帧之间的深度差分轮廓（DDS）在三个正交平面上的投影。提出的时空多融合特征的大小减少了码书中的一个码矢量，这是由矢量量化方法产生的。然后，它训练的隐马尔可夫模型（HMM）的多融合特征的代码向量和识别分割的人体活动的前向斑点计划使用训练的HMM为基础的人体活动分类器。在IM-Daily-DepthActivity、MSRAction 3D和MSRDailyActivity 3D等三个具有挑战性的深度视频数据集上的实验结果表明，使用所提出的多融合特征的在线HAR方法在识别准确性方面优于最先进的HAR方法。(C)2016爱思唯尔有限公司版权所有
The recently developed depth imaging technologies have provided new directions for human activity recognition (HAR) without attaching optical markers or any other motion sensors to human body parts. In this paper, we propose novel multi-fused features for online human activity recognition (HAR) system that recognizes human activities from continuous sequences of depth map. The proposed online HAR system segments human depth silhouettes using temporal human motion information as well as it obtains human skeleton joints using spatiotemporal human body information. Then, it extracts the spatiotemporal multi-fused features that concatenate four skeleton joint features and one body shape feature. Skeleton joint features include the torso-based distance feature (DT), the key joint-based distance feature (DK), the spatiotemporal magnitude feature (M) and the spatiotemporal directional angle feature (theta). The body shape feature called HOG-DDS represents the projections of the depth differential silhouettes (DDS) between two consecutive frames onto three orthogonal planes by the histogram of oriented gradients (HOG) format. The size of the proposed spatiotemporal multi-fused feature is reduced by a code vector in the code book which is generated by vector quantization method. Then, it trains the hidden Markov model (HMM) with the code vectors of the multi-fused features and recognizes the segmented human activity by the forward spotting scheme using the trained HMM-based human activity classifiers. The experimental results on three challenging depth video datasets such as IM-Daily-DepthActivity, MSRAction3D and MSRDailyActivity3D demonstrate that the proposed online HAR method using the proposed multi-fused features outperforms the state-of-the-art HAR methods in terms of recognition accuracy. (C) 2016 Elsevier Ltd. All rights reserved.