FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction
FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction
复制标题
DOI：
10.1109/tpami.2023.3307338
复制
发表时间：
2021-11
影响因子：
23.6
通讯作者：
Hao Zhu;Haotian Yang;Longwei Guo;Yiding Zhang;Yanru Wang;Mingkai Huang;Qiu Shen;Ruigang Yang;Xun Cao
中科院分区：
文献类型：
--
作者：
Hao Zhu;Haotian Yang;Longwei Guo;Yiding Zhang;Yanru Wang;Mingkai Huang;Qiu Shen;Ruigang Yang;Xun Cao
在这篇文章中，我们提出了一个大规模的详细的三维人脸数据集，FaceScape，和相应的基准来评估单视图面部三维重建。通过对FaceScape数据进行训练，提出了一种新算法，可以从单个图像输入预测复杂的可装配3D人脸模型。FaceScape数据集发布了16，940张纹理3D人脸，从847名受试者中捕获，每个人都有20种特定的表情。3D模型包含也被处理为拓扑均匀的孔隙级面部几何形状。这些精细的3D面部模型可以表示为用于粗略形状的3D可变形模型和用于详细几何形状的位移图。利用大规模和高精度数据集的优势，进一步提出了一种新的算法，使用深度神经网络来学习特定于表情的动态细节。学习的关系作为我们的3D人脸预测系统的基础，从一个单一的图像输入。与大多数以前的方法不同，我们预测的3D模型在不同的表达式下具有高度详细的几何形状。我们还使用FaceScape数据来生成野外和实验室基准，以评估最近的单视图人脸重建方法。从摄像机位姿和焦距两个维度对精度进行了分析，给出了一个真实而全面的评价，同时也揭示了新的挑战。前所未有的数据集，基准和代码已向公众发布用于研究目的。
In this article, we present a large-scale detailed 3D face dataset, FaceScape, and the corresponding benchmark to evaluate single-view facial 3D reconstruction. By training on FaceScape data, a novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input. FaceScape dataset releases 16,940 textured 3D faces, captured from 847 subjects and each with 20 specific expressions. The 3D models contain the pore-level facial geometry that is also processed to be topologically uniform. These fine 3D facial models can be represented as a 3D morphable model for coarse shapes and displacement maps for detailed geometry. Taking advantage of the large-scale and high-accuracy dataset, a novel algorithm is further proposed to learn the expression-specific dynamic details using a deep neural network. The learned relationship serves as the foundation of our 3D face prediction system from a single image input. Different from most previous methods, our predicted 3D models are riggable with highly detailed geometry under different expressions. We also use FaceScape data to generate the in-the-wild and in-the-lab benchmark to evaluate recent methods of single-view face reconstruction. The accuracy is reported and analyzed on the dimensions of camera pose and focal length, which provides a faithful and comprehensive evaluation and reveals new challenges. The unprecedented dataset, benchmark, and code have been released to the public for research purpose.