IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images
IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images
复制标题
DOI：
10.1109/cvpr52688.2022.00548
复制
发表时间：
2022-04
期刊：
影响因子：
--
通讯作者：
Kai Zhang;Fujun Luan;Zhengqi Li;Noah Snavely
中科院分区：
文献类型：
--
作者：
Kai Zhang;Fujun Luan;Zhengqi Li;Noah Snavely
我们提出了一个名为IRON的神经逆渲染管道，它对光度图像进行操作，并以三角形网格和材料纹理的格式输出高质量的3D内容，这些内容可以在现有的图形管道中轻松部署。我们的方法在优化过程中采用神经表示几何形状作为符号距离场（SDF）和材料，以享受其灵活性和紧凑性，并为神经SDF提供混合优化方案：首先，使用体积辐射场方法进行优化以恢复正确的拓扑结构，然后使用基于边缘感知物理的表面渲染进一步优化几何形状细化和材料和照明的解纠缠。在第二阶段，我们也从基于网格的可微分绘制中得到启发，设计了一种新的神经SDF边缘采样算法，以进一步提高性能。我们表明，我们的IRON实现了显着更好的逆渲染质量相比，以前的作品。
We propose a neural inverse rendering pipeline called IRON that operates on photometric images and outputs high-quality 3D content in the format of triangle meshes and material textures readily deployable in existing graphics pipelines. Our method adopts neural representations for geometry as signed distance fields (SDFs) and materials during optimization to enjoy their flexibility and compactness, and features a hybrid optimization scheme for neural SDFs: first, optimize using a volumetric radiance field approach to recover correct topology, then optimize further using edgeaware physics-based surface rendering for geometry refinement and disentanglement of materials and lighting. In the second stage, we also draw inspiration from mesh-based differentiable rendering, and design a novel edge sampling algorithm for neural SDFs to further improve performance. We show that our IRON achieves significantly better inverse rendering quality compared to prior works.