MirrorNet: A Deep Reflective Approach to 2D Pose Estimation for Single-Person Images
MirrorNet: A Deep Reflective Approach to 2D Pose Estimation for Single-Person Images
复制标题
DOI：
10.2197/ipsjjip.29.406
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Takayuki Nakatsuka;Kazuyoshi Yoshii;Yuki Koyama;Satoru Fukayama;Masataka Goto;S. Morishima
中科院分区：
文献类型：
--
作者：
Takayuki Nakatsuka;Kazuyoshi Yoshii;Yuki Koyama;Satoru Fukayama;Masataka Goto;S. Morishima
提出了一种基于统计的人体图像二维姿态估计方法。基于深度识别（图像到姿势）模型的标准监督方法的主要问题是，它通常会产生解剖学上不可信的姿势，并且其性能受到配对数据量的限制。为了解决这些问题，我们提出了一种半监督的方法，可以有效地利用图像的姿态注释和没有姿态注释。具体而言，我们通过将来自姿势特征的姿势的深度生成模型与来自姿势和图像特征的图像的深度生成模型相集成，来制定姿势和图像的分层生成模型。然后，我们介绍了一个深度识别模型，从图像中推断姿势。给定图像作为观察数据，这些模型可以以分层变分自动编码（图像到姿态到特征到姿态到图像）的方式联合训练。实验结果表明，所提出的自适应结构使姿态估计在解剖学上合理，通过集成识别模型和生成模型以及输入无标注图像，姿态估计性能得到了提高。
: This paper proposes a statistical approach to 2D pose estimation from human images. The main problems with the standard supervised approach, which is based on a deep recognition (image-to-pose) model, are that it often yields anatomically implausible poses, and its performance is limited by the amount of paired data. To solve these problems, we propose a semi-supervised method that can make e ff ective use of images with and without pose annotations. Specifically, we formulate a hierarchical generative model of poses and images by integrating a deep generative model of poses from pose features with that of images from poses and image features. We then introduce a deep recognition model that infers poses from images. Given images as observed data, these models can be trained jointly in a hierarchical variational autoencoding (image-to-pose-to-feature-to-pose-to-image) manner. The results of experiments show that the proposed reflective architecture makes estimated poses anatomically plausible, and the pose estimation performance is improved by integrating the recognition and generative models and also by feeding non-annotated images.