Neural Fields for Structured Lighting
Neural Fields for Structured Lighting
复制标题
DOI：
10.1109/iccv51070.2023.00325
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Aarrushi Shandilya;Benjamin Attal;Christian Richardt;James Tompkin;Matthew O’Toole
中科院分区：
文献类型：
--
作者：
Aarrushi Shandilya;Benjamin Attal;Christian Richardt;James Tompkin;Matthew O’Toole
我们提出了一个图像形成模型和优化过程，结合神经辐射场和结构光成像的优点。现有的深度监督神经模型依赖于深度传感器来准确捕捉场景的几何形状。然而，这些传感器恢复的深度图可能容易出错，甚至完全失败。更有原则的方法是显式地对原始结构光图像本身进行建模，而不是依赖于结构光系统处理后的深度图的保真度。我们提出的方法能够估计高保真深度图，包括具有复杂材料属性的对象（例如，部分透明的表面）。除了计算深度之外，原始结构光图像还赋予其他有用的辐射线索，这使得能够预测表面法线并根据直接、间接和环境分量来分解场景外观。我们评估我们的框架定量和定性的范围内的真实的和合成的场景，并分解成其组成部分的新颖的观点的场景。
We present an image formation model and optimization procedure that combines the advantages of neural radiance fields and structured light imaging. Existing depth-supervised neural models rely on depth sensors to accurately capture the scene’s geometry. However, the depth maps recovered by these sensors can be prone to error, or even fail outright. Instead of depending on the fidelity of processed depth maps from a structured light system, a more principled approach is to explicitly model the raw structured light images themselves. Our proposed approach enables the estimation of high-fidelity depth maps, including for objects with complex material properties (e.g., partially-transparent surfaces). Besides computing depth, the raw structured light images also confer other useful radiometric cues, which enable predicting surface normals and decomposing scene appearance in terms of a direct, indirect, and ambient component. We evaluate our framework quantitatively and qualitatively on a range of real and synthetic scenes, and decompose scenes into their constituent components for novel views.