PANDORA: Polarization-Aided Neural Decomposition Of Radiance
PANDORA: Polarization-Aided Neural Decomposition Of Radiance
复制标题
DOI：
10.48550/arxiv.2203.13458
复制
发表时间：
2022-03
期刊：
影响因子：
--
通讯作者：
Akshat Dave;Yongyi Zhao;A. Veeraraghavan
中科院分区：
文献类型：
--
作者：
Akshat Dave;Yongyi Zhao;A. Veeraraghavan
从多幅图像重建物体的几何形状和外观，也称为逆绘制，是计算机图形学和视觉中的一个基本问题。逆渲染是固有的不适定，因为捕获的图像是一个复杂的功能未知的照明条件，材料属性和场景几何。最近的进展表示场景属性为基于坐标的神经网络，促进神经逆渲染，从而产生令人印象深刻的几何重建和新颖的视图合成。我们的关键见解是，偏振是神经逆渲染的有用线索，因为偏振强烈依赖于表面法线，并且对于漫反射和镜面反射是不同的。随着商品、片上偏振传感器的出现，捕获偏振已经变得实用。因此，我们提出了PANDORA，一种基于隐式神经表示的极化逆绘制方法。从物体的多视图偏振图像中，PANDORA联合提取物体的3D几何形状，将出射辐射分为漫射和镜面反射，并估计入射到物体上的照明。我们表明，PANDORA优于国家的最先进的辐射分解技术。PANDORA输出无纹理伪影的干净表面重建，精确地建模强镜面反射，并在实际的非结构化场景下估计照明。
Reconstructing an object's geometry and appearance from multiple images, also known as inverse rendering, is a fundamental problem in computer graphics and vision. Inverse rendering is inherently ill-posed because the captured image is an intricate function of unknown lighting conditions, material properties and scene geometry. Recent progress in representing scene properties as coordinate-based neural networks have facilitated neural inverse rendering resulting in impressive geometry reconstruction and novel-view synthesis. Our key insight is that polarization is a useful cue for neural inverse rendering as polarization strongly depends on surface normals and is distinct for diffuse and specular reflectance. With the advent of commodity, on-chip, polarization sensors, capturing polarization has become practical. Thus, we propose PANDORA, a polarimetric inverse rendering approach based on implicit neural representations. From multi-view polarization images of an object, PANDORA jointly extracts the object's 3D geometry, separates the outgoing radiance into diffuse and specular and estimates the illumination incident on the object. We show that PANDORA outperforms state-of-the-art radiance decomposition techniques. PANDORA outputs clean surface reconstructions free from texture artefacts, models strong specularities accurately and estimates illumination under practical unstructured scenarios.