Deep Photometric Stereo Networks for Determining Surface Normal and Reflectances
Deep Photometric Stereo Networks for Determining Surface Normal and Reflectances
复制标题
用于确定表面法线和反射率的深度光度立体网络
DOI：
10.1109/tpami.2020.3005219
复制
发表时间：
2022-01-01
影响因子：
23.6
通讯作者：
Matsushita, Yasuyuki
中科院分区：
文献类型：
--
作者：
Santo, Hiroaki;Samejima, Masaki;Matsushita, Yasuyuki
本文提出了一种基于深度学习的光度立体方法。在光度立体的主要困难之一是设计一个适当的反射模型，既能够代表现实世界的反射率和计算易处理的推导表面法线。与依赖于简化的参数图像形成模型（例如Lambert模型）的先前的光度立体方法不同，所提出的方法旨在使用深度神经网络在复杂反射率观测与表面法线之间建立灵活的映射。此外，所提出的方法预测的反射率，这使我们能够了解表面材料和渲染场景在任意光照条件下。因此，我们提出了一种深度光度立体网络（DPSN），它在不同的光线方向下进行反射观测，并以每像素的方式推断表面法线和反射率。为了使DPSN适用于现实世界的场景，测量的BRDF数据集（MERL BRDF数据集）已被用于训练网络。使用模拟和真实世界的场景评估表明，该方法在估计表面法线和反射率的有效性。
This article presents a photometric stereo method based on deep learning. One of the major difficulties in photometric stereo is designing an appropriate reflectance model that is both capable of representing real-world reflectances and computationally tractable for deriving surface normal. Unlike previous photometric stereo methods that rely on a simplified parametric image formation model, such as the Lambert's model, the proposed method aims at establishing a flexible mapping between complex reflectance observations and surface normal using a deep neural network. In addition, the proposed method predicts the reflectance, which allows us to understand surface materials and to render the scene under arbitrary lighting conditions. As a result, we propose a deep photometric stereo network (DPSN) that takes reflectance observations under varying light directions and infers the surface normal and reflectance in a per-pixel manner. To make the DPSN applicable to real-world scenes, a dataset of measured BRDFs (MERL BRDF dataset) has been used for training the network. Evaluation using simulation and real-world scenes shows the effectiveness of the proposed approach in estimating both surface normal and reflectances.