Polarimetric Multi-View Inverse Rendering
Polarimetric Multi-View Inverse Rendering
复制标题
DOI：
10.1109/tpami.2022.3232211
复制
发表时间：
2020-07
影响因子：
23.6
通讯作者：
Jinyu Zhao;Yusuke Monno;M. Okutomi
中科院分区：
文献类型：
--
作者：
Jinyu Zhao;Yusuke Monno;M. Okutomi
偏振相机具有用于3D重建的巨大潜力，因为反射光的偏振角（AoP）和偏振度（DoP）与物体的表面法线相关。在本文中，我们提出了一种新的三维重建方法称为偏振多视图逆渲染（偏振MVIR），有效地利用几何，光度，和偏振线索提取输入多视图彩色偏振图像。我们首先估计相机姿势和一个初始的3D模型的几何重建与一个标准的结构从运动和多视图立体管道。然后，我们通过使用多视图RGB，AoP和DoP图像优化光度渲染误差和偏振误差来改进初始模型，其中我们提出了一种新的偏振成本函数，该函数能够有效约束每个顶点的估计表面法线，同时考虑从AoP测量中揭示的四个可能的模糊方位角。该方法基于偏振度测量值有效地确定了极化代价的权重，即极化信息的可靠性。使用合成和真实的数据的实验结果表明，我们的偏振MVIR可以重建一个详细的三维形状，而无需假设特定的表面材料和照明条件。
A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) and the degree of polarization (DoP) of reflected light are related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric rendering errors and polarimetric errors using multi-view RGB, AoP, and DoP images, where we propose a novel polarimetric cost function that enables an effective constraint on the estimated surface normal of each vertex, while considering four possible ambiguous azimuth angles revealed from the AoP measurement. The weight for the polarimetric cost is effectively determined based on the DoP measurement, which is regarded as the reliability of polarimetric information. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific surface material and lighting condition.