End-to-End 3D Face Reconstruction with Expressions and Specular Albedos from Single In-the-wild Images
End-to-End 3D Face Reconstruction with Expressions and Specular Albedos from Single In-the-wild Images
复制标题
DOI：
10.1145/3503161.3547800
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Qixin Deng;B. Le;Aobo Jin;Z. Deng
中科院分区：
文献类型：
--
作者：
Qixin Deng;B. Le;Aobo Jin;Z. Deng
从野外人脸图像中恢复3D人脸模型有许多潜在的应用。然而，在现实中正确建模复杂的照明效果，包括镜面照明，阴影和遮挡，从单一的野外人脸图像仍然被认为是一个广泛开放的研究挑战。在本文中，我们提出了一个基于卷积神经网络的框架，从野外的单个图像中回归人脸模型。输出的人脸模型包括密集的三维形状、头部姿态、表情、漫射反照率、镜面反照率以及相应的光照条件。我们的方法使用新颖的混合损失函数来分离面部形状身份、表情、姿势、反照率和光照。除了精心设计的消融研究外，我们还进行了直接比较实验，以表明我们的方法在定量和定性上都优于目前最先进的方法。
Recovering 3D face models from in-the-wild face images has numerous potential applications. However, properly modeling complex lighting effects in reality, including specular lighting, shadows, and occlusions, from a single in-the-wild face image is still considered as a widely open research challenge. In this paper, we propose a convolutional neural network based framework to regress the face model from a single image in the wild. The outputted face model includes dense 3D shape, head pose, expression, diffuse albedo, specular albedo, and the corresponding lighting conditions. Our approach uses novel hybrid loss functions to disentangle face shape identities, expressions, poses, albedos, and lighting. Besides a carefully-designed ablation study, we also conduct direct comparison experiments to show that our method can outperform state-of-art methods both quantitatively and qualitatively.