Deep Photometric Stereo for Non-Lambertian Surfaces
Deep Photometric Stereo for Non-Lambertian Surfaces
复制标题
非朗伯表面的深度光度立体
DOI：
10.1109/tpami.2020.3005397
复制
发表时间：
2022-01-01
影响因子：
23.6
通讯作者：
Wong, Kwan-Yee K.
中科院分区：
文献类型：
--
作者：
Chen, Guanying;Han, Kai;Wong, Kwan-Yee K.
本文研究了基于深度学习的非朗伯曲面在标定和未标定两种情况下的光度立体问题。我们首先介绍了一种用于标定光度立体的完全卷积深度网络，我们称之为PS-FCN。不同于传统方法采用简化的反射率模型来处理问题，该方法直接学习从反射率观测值到表面法线的映射，并且能够处理具有一般和未知各向同性反射率的表面。在测试时，PS-FCN将任意数量的图像及其关联的灯光方向作为输入，并在快速前馈过程中预测场景的曲面法线贴图。为了处理光方向未知的未标定场景，我们引入了一种新的卷积网络，称为LCNet，用于从输入图像估计光方向。然后，将估计的光方向和输入图像馈送到PS-FCN以确定曲面法线。我们的方法不需要预先定义的一组光方向，并且可以以与顺序无关的方式处理多幅图像。在合成数据集和真实数据集上对我们的方法进行的全面评估表明，它在校准和未校准场景中的性能都优于最先进的方法。
This paper addresses the problem of photometric stereo, in both calibrated and uncalibrated scenarios, for non-Lambertian surfaces based on deep learning. We first introduce a fully convolutional deep network for calibrated photometric stereo, which we call PS-FCN. Unlike traditional approaches that adopt simplified reflectance models to make the problem tractable, our method directly learns the mapping from reflectance observations to surface normal, and is able to handle surfaces with general and unknown isotropic reflectance. At test time, PS-FCN takes an arbitrary number of images and their associated light directions as input and predicts a surface normal map of the scene in a fast feed-forward pass. To deal with the uncalibrated scenario where light directions are unknown, we introduce a new convolutional network, named LCNet, to estimate light directions from input images. The estimated light directions and the input images are then fed to PS-FCN to determine the surface normals. Our method does not require a pre-defined set of light directions and can handle multiple images in an order-agnostic manner. Thorough evaluation of our approach on both synthetic and real datasets shows that it outperforms state-of-the-art methods in both calibrated and uncalibrated scenarios.