Detailed Avatar Recovery From Single Image
Detailed Avatar Recovery From Single Image
复制标题
DOI：
10.1109/tpami.2021.3102128
复制
发表时间：
2021-08
影响因子：
23.6
通讯作者：
Hao Zhu;X. Zuo;Haotian Yang;Sen Wang;Xun Cao;Ruigang Yang
中科院分区：
文献类型：
--
作者：
Hao Zhu;X. Zuo;Haotian Yang;Sen Wang;Xun Cao;Ruigang Yang
提出了一种从单幅图像中恢复细节化身的新框架。由于人体形状、身体姿势、纹理和视角等因素的变化，这是一项具有挑战性的任务。现有方法通常尝试使用缺乏表面细节的基于参数的模板来恢复人体形状。因此，由此产生的体型似乎没有穿衣服。在本文中，我们提出了一种新的基于学习的框架，它结合了参数模型的健壮性和自由形式的3D变形的灵活性。我们使用深度神经网络在分层网格变形(HMD)框架中细化3D形状，利用来自身体关节、轮廓和每像素阴影信息的约束。我们的方法可以恢复人体的细节形状，在蒙皮模型之外具有完整的纹理。实验表明，我们的方法在2D IOU数和3D度量距离方面都取得了更好的精度，性能优于以往的方法。
This paper presents a novel framework to recover detailed avatar from a single image. It is a challenging task due to factors such as variations in human shapes, body poses, texture, and viewpoints. Prior methods typically attempt to recover the human body shape using a parametric-based template that lacks the surface details. As such resulting body shape appears to be without clothing. In this paper, we propose a novel learning-based framework that combines the robustness of the parametric model with the flexibility of free-form 3D deformation. We use the deep neural networks to refine the 3D shape in a Hierarchical Mesh Deformation (HMD) framework, utilizing the constraints from body joints, silhouettes, and per-pixel shading information. Our method can restore detailed human body shapes with complete textures beyond skinned models. Experiments demonstrate that our method has outperformed previous state-of-the-art approaches, achieving better accuracy in terms of both 2D IoU number and 3D metric distance.