Multimodal Deep Autoencoder for Human Pose Recovery
Multimodal Deep Autoencoder for Human Pose Recovery
复制标题
用于人体姿势恢复的多模态深度自动编码器
DOI：
10.1109/tip.2015.2487860
复制
发表时间：
2015-12-01
影响因子：
10.6
通讯作者：
Wang, Meng
中科院分区：
文献类型：
--
作者：
Hong, Chaoqun;Yu, Jun;Wang, Meng
基于视频的人体姿势恢复通常是通过使用图像特征检索相关姿势来进行的。在检索过程中，大多数传统方法都假设2D图像和3D姿态之间的映射是线性的。然而，它们之间的关系本质上是非线性的，这限制了这些方法的恢复性能。本文提出了一种基于多层深度神经网络的非线性映射姿态恢复方法。它是基于多模式融合和反向传播深度学习的特征提取。在多模式融合中，我们构造了具有低阶表示的超图拉普拉斯。这样，我们通过超图拉普拉斯矩阵的标准特征分解得到了统一的特征描述。在反向传播深度学习中，我们学习了从2D图像到3D姿态的非线性映射，并进行了参数微调。在三个数据集上的实验结果表明，恢复误差降低了20%-25%，证明了该方法的有效性。
Video-based human pose recovery is usually conducted by retrieving relevant poses using image features. In the retrieving process, the mapping between 2D images and 3D poses is assumed to be linear in most of the traditional methods. However, their relationships are inherently non-linear, which limits recovery performance of these methods. In this paper, we propose a novel pose recovery method using non-linear mapping with multi-layered deep neural network. It is based on feature extraction with multimodal fusion and back-propagation deep learning. In multimodal fusion, we construct hypergraph Laplacian with low-rank representation. In this way, we obtain a unified feature description by standard eigen-decomposition of the hypergraph Laplacian matrix. In back-propagation deep learning, we learn a non-linear mapping from 2D images to 3D poses with parameter fine-tuning. The experimental results on three data sets show that the recovery error has been reduced by 20%-25%, which demonstrates the effectiveness of the proposed method.