MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
复制标题
DOI：
10.1109/iccv.2017.401
复制
发表时间：
2017-03
期刊：
影响因子：
--
通讯作者：
A. Tewari;M. Zollhöfer;Hyeongwoo Kim;Pablo Garrido;Florian Bernard;P. Pérez;C. Theobalt
中科院分区：
文献类型：
--
作者：
A. Tewari;M. Zollhöfer;Hyeongwoo Kim;Pablo Garrido;Florian Bernard;P. Pérez;C. Theobalt
在这项工作中，我们提出了一种新的基于模型的深度卷积自动编码器，它解决了从单个原始彩色图像重建3D人脸的高度挑战性问题。为此，我们将联合收割机卷积编码器网络与专家设计的生成模型（作为解码器）相结合。核心创新是可微分参数解码器，它基于生成模型分析地封装图像形成。我们的解码器作为输入的代码向量与精确定义的语义含义，编码详细的人脸姿态，形状，表情，皮肤反射和场景照明。由于这种将基于CNN的人脸重建与基于模型的人脸重建相结合的新方法，基于CNN的编码器可以学习从单个单目输入图像中提取语义上有意义的参数。这是第一次，CNN编码器和专家设计的生成模型可以以无监督的方式进行端到端的训练，这使得在非常大的（未标记的）真实的世界数据上进行训练变得可行。所获得的重建相比，目前国家的最先进的方法在质量和丰富的代表性。
In this work we propose a novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image. To this end, we combine a convolutional encoder network with an expert-designed generative model that serves as decoder. The core innovation is the differentiable parametric decoder that encapsulates image formation analytically based on a generative model. Our decoder takes as input a code vector with exactly defined semantic meaning that encodes detailed face pose, shape, expression, skin reflectance and scene illumination. Due to this new way of combining CNN-based with model-based face reconstruction, the CNN-based encoder learns to extract semantically meaningful parameters from a single monocular input image. For the first time, a CNN encoder and an expert-designed generative model can be trained end-to-end in an unsupervised manner, which renders training on very large (unlabeled) real world data feasible. The obtained reconstructions compare favorably to current state-of-the-art approaches in terms of quality and richness of representation.