Towards Learning-based Inverse Subsurface Scattering
Towards Learning-based Inverse Subsurface Scattering
复制标题
DOI：
10.1109/iccp48838.2020.9105209
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Chengqian Che;Fujun Luan;Shuang Zhao;Kavita Bala;Ioannis Gkioulekas
中科院分区：
文献类型：
--
作者：
Chengqian Che;Fujun Luan;Shuang Zhao;Kavita Bala;Ioannis Gkioulekas
给定半透明物体的图像，形状和光照未知，我们的目标是利用学习来推断控制物体内部光的次表面散射的光学参数。我们引入了一种新的架构，即逆传输网络（ITN），旨在通过将编码器网络与能够根据散射材料参数估计图像导数的物理精确、可微的蒙特卡罗渲染器连接起来，提高编码器网络对未见场景的泛化。在训练过程中，这种组合迫使编码器网络预测的参数不仅要匹配真值，而且还要重现输入图像。在测试期间，单独使用编码器网络，而不使用渲染器，从单个输入图像预测材料参数。从辐射传输的物理学中获得见解，我们还使用材料参数化来帮助减少由于散射参数空间中的模糊性而导致的估计误差。最后，我们用像素权重图来增加训练损失，这些权重图强调图像中最能提供潜在散射参数信息的部分。我们证明，这种组合使神经网络能够比传统网络更好地泛化到具有完全看不见的几何形状和光照的场景，平均降低了38.06%的参数误差。
Given images of translucent objects, of unknown shape and lighting, we aim to use learning to infer the optical parameters controlling subsurface scattering of light inside the objects. We introduce a new architecture, the inverse transport network (ITN), that aims to improve generalization of an encoder network to unseen scenes, by connecting it with a physically-accurate, differentiable Monte Carlo renderer capable of estimating image derivatives with respect to scattering material parameters. During training, this combination forces the encoder network to predict parameters that not only match groundtruth values, but also reproduce input images. During testing, the encoder network is used alone, without the renderer, to predict material parameters from a single input image. Drawing insights from the physics of radiative transfer, we additionally use material parameterizations that help reduce estimation errors due to ambiguities in the scattering parameter space. Finally, we augment the training loss with pixelwise weight maps that emphasize the parts of the image most informative about the underlying scattering parameters. We demonstrate that this combination allows neural networks to generalize to scenes with completely unseen geometries and illuminations better than traditional networks, with 38.06% reduced parameter error on average.