Temporal Consistency Loss for High Resolution Textured and Clothed 3D Human Reconstruction from Monocular Video
Temporal Consistency Loss for High Resolution Textured and Clothed 3D Human Reconstruction from Monocular Video
复制标题
DOI：
10.1109/cvprw53098.2021.00197
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Akin Caliskan;A. Mustafa;A. Hilton
中科院分区：
文献类型：
--
作者：
Akin Caliskan;A. Mustafa;A. Hilton
我们提出了一种新方法，可以从单眼视频中学习对着装人员进行时间一致的 3D 重建。最近使用体积、隐式或参数人体形状模型从单眼视频进行 3D 人体重建的方法，产生每帧重建，在应用于视频时给出时间不一致的输出和有限的性能。在本文中，我们介绍了一种学习时间一致特征的方法，用于从单目视频中对穿着 3D 人体序列进行纹理重建，并提出了两项进展：一种新颖的时间一致性损失函数；以及用于从 2D 图像和粗略 3D 几何结构进行隐式 3D 重建的混合表示学习。所提出的进展提高了单目视频 3D 重建和纹理预测的时间一致性和准确性。对人体图像的综合比较性能评估表明，所提出的方法显着优于最先进的基于学习的单图像 3D 人体形状估计方法，在重建精度、完整性、质量和时间一致性方面实现了显着提高。
We present a novel method to learn temporally consistent 3D reconstruction of clothed people from a monocular video. Recent methods for 3D human reconstruction from monocular video using volumetric, implicit or parametric human shape models, produce per frame reconstructions giving temporally inconsistent output and limited performance when applied to video. In this paper we introduce an approach to learn temporally consistent features for textured reconstruction of clothed 3D human sequences from monocular video by proposing two advances: a novel temporal consistency loss function; and hybrid representation learning for implicit 3D reconstruction from 2D images and coarse 3D geometry. The proposed advances improve the temporal consistency and accuracy of both the 3D reconstruction and texture prediction from a monocular video. Comprehensive comparative performance evaluation on images of people demonstrates that the proposed method significantly outperforms the state-of-the-art learning-based single image 3D human shape estimation approaches achieving significant improvement of reconstruction accuracy, completeness, quality and temporal consistency.