Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising
Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising
复制标题
DOI：
--
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
J. Hasselgren;Nikolai Hofmann;Jacob Munkberg
中科院分区：
文献类型：
--
作者：
J. Hasselgren;Nikolai Hofmann;Jacob Munkberg
可差分渲染的最新进展使得能够从多视点图像高质量地重建3D场景。大多数方法依赖于简单的渲染算法：预先过滤的直接照明或已学习的辐照度表示法。我们表明，一个更真实的阴影模型，结合光线跟踪和蒙特卡罗积分，大大改善了分解为形状，材料。不幸的是，蒙特卡罗积分提供的估计具有显著的噪声，即使在大样本计数的情况下也是如此，这使得基于梯度的逆渲染非常具有挑战性。为了解决这个问题，我们在一个新的逆向渲染流水线中结合了多个重要性采样和去噪。这大大提高了收敛速度，并在低样本数下实现了基于梯度的优化。我们提出了一种有效的方法来联合重建几何体(显式三角形网格)、材质和光照，与以前的工作相比，该方法显著地改善了材质和灯光的分离。我们认为，去噪可以成为高质量反向渲染流水线的组成部分。
Recent advances in differentiable rendering have enabled high-quality reconstruction of 3D scenes from multi-view images. Most methods rely on simple rendering algorithms: pre-filtered direct lighting or learned representations of irradiance. We show that a more realistic shading model, incorporating ray tracing and Monte Carlo integration, substantially improves decomposition into shape, materials. Unfortunately, Monte Carlo integration provides estimates with significant noise, even at large sample counts, which makes gradient-based inverse rendering very challenging. To address this, we incorporate multiple importance sampling and denoising in a novel inverse rendering pipeline. This substantially improves convergence and enables gradient-based optimization at low sample counts. We present an efficient method to jointly reconstruct geometry (explicit triangle meshes), materials, and lighting, which substantially improves material and light separation compared to previous work. We argue that denoising can become an integral part of high quality inverse rendering pipelines.