2D Pose-Based Real-Time Human Action Recognition With Occlusion-Handling
2D Pose-Based Real-Time Human Action Recognition With Occlusion-Handling
复制标题
DOI：
10.1109/tmm.2019.2944745
复制
发表时间：
2020-06-01
影响因子：
7.3
通讯作者：
Naqvi, Syed Mohsen
中科院分区：
文献类型：
--
作者：
Angelini, Federico;Fu, Zeyu;Naqvi, Syed Mohsen
面向CCTV的人体动作识别（HAR）仍然是一个具有挑战性的问题。由于深度学习数据需求与基于CCTV的框架在数据记录设备方面所能提供的差距差距，现实场景的HAR实现是困难的。我们建议通过利用OpenPose提供的人体姿势来缩小这一差距，OpenPose已经被证明是跟踪应用中类似CCTV记录的有效检测器。因此，在这项工作中，我们首先提出了一种新的基于2D姿态的姿态级HAR方法--XPose。XPose从身体姿势中提取低级和高级特征，并将其提供给长短期记忆神经网络和一维卷积神经网络进行分类。我们还提供了一个新的数据集，名为ISLD，逼真的姿态级HAR在一个类似CCTV的环境中，记录在智能传感实验室。XPose在ISLD上进行了广泛的测试，包括多个实验设置，例如数据集增强和跨数据集设置，以及为HAR修改其他现有数据集。XPose在准确性、对遮挡和丢失数据的高度鲁棒性以及在现实世界应用中的实际实施方面实现了最先进的性能。
Human Action Recognition (HAR) for CCTV-oriented applications is still a challenging problem. Real-world scenarios HAR implementations is difficult because of the gap between Deep Learning data requirements and what the CCTV-based frameworks can offer in terms of data recording equipments. We propose to reduce this gap by exploiting human poses provided by the OpenPose, which has been already proven to be an effective detector in CCTV-like recordings for tracking applications. Therefore, in this work, we first propose ActionXPose: a novel 2D pose-based approach for pose-level HAR. ActionXPose extracts low- and high-level features from body poses which are provided to a Long Short-Term Memory Neural Network and a 1D Convolutional Neural Network for the classification. We also provide a new dataset, named ISLD, for realistic pose-level HAR in a CCTV-like environment, recorded in the Intelligent Sensing Lab. ActionXPose is extensively tested on ISLD under multiple experimental settings, e.g. Dataset Augmentation and Cross-Dataset setting, as well as revising other existing datasets for HAR. ActionXPose achieves state-of-the-art performance in terms of accuracy, very high robustness to occlusions and missing data, and promising results for practical implementation in real-world applications.