From Local to Holistic: Self-supervised Single Image 3D Face Reconstruction Via Multi-level Constraints
From Local to Holistic: Self-supervised Single Image 3D Face Reconstruction Via Multi-level Constraints
复制标题
DOI：
10.1109/iros47612.2022.9982284
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Yawen Lu;M. Sarkis;N. Bi;G. Lu
中科院分区：
文献类型：
--
作者：
Yawen Lu;M. Sarkis;N. Bi;G. Lu
单图3D面部重建具有准确的几何细节是一项至关重要且具有挑战性的任务，因为面部表面的外观相似，并且器官中的细节细节。在这项工作中，我们从单个图像中引入了一种自我监管的3D面部重建方法，该方法可以在不同的相机设置下恢复详细的纹理。拟议的网络在训练阶段学习了立体声面部图像的高质量差异图，而仅需要单个面图像来在实际应用中生成3D模型。为了恢复每个器官和面部表面的细节，该框架引入了面部地标空间一致性，以限制面部器官的局部点水平和分割方案的面部恢复学习过程，以在器官级别限制对应关系。面部形状和纹理将通过基于不同的光照明和阴影信息建立整体约束来进一步完善。与最先进的3DMM和基于几何形状的重建算法相比，提出的学习框架可以在定量和定性上恢复更准确的3D面部细节。
Single image 3D face reconstruction with accurate geometric details is a critical and challenging task due to the similar appearance on the face surface and fine details in organs. In this work, we introduce a self-supervised 3D face reconstruction approach from a single image that can recover detailed textures under different camera settings. The proposed network learns high-quality disparity maps from stereo face images during the training stage, while just a single face image is required to generate the 3D model in real applications. To recover fine details of each organ and facial surface, the framework introduces facial landmark spatial consistency to constrain the face recovering learning process in local point level and segmentation scheme on facial organs to constrain the correspondences at the organ level. The face shape and textures will further be refined by establishing holistic constraints based on the varying light illumination and shading information. The proposed learning framework can recover more accurate 3D facial details both quantitatively and qualitatively compared with state-of-the-art 3DMM and geometry-based reconstruction algorithms based on a single image.