Single-Shot Neural Relighting and SVBRDF Estimation
Single-Shot Neural Relighting and SVBRDF Estimation
复制标题
DOI：
10.1007/978-3-030-58529-7_6
复制
发表时间：
2020
影响因子：
4.4
通讯作者：
S. Sang;Manmohan Chandraker
中科院分区：
文献类型：
--
作者：
S. Sang;Manmohan Chandraker
我们提出了一种新的物理驱动的深度网络，用于关节形状和材料估计，以及在新的照明条件下重新照明，使用由移动的手机摄像头捕获的单个图像。我们基于物理的建模利用了在大规模合成数据集上训练的深度级联架构，该数据集由具有microfacet SVBRDF的复杂形状组成。与之前在逆渲染之后训练渲染层的工作相比，我们提出了深度特征共享和联合训练，可以在两个任务之间传递见解，从而在重建和重新照明方面都取得了显着的改进。我们在大量的定性和定量实验中证明，我们的网络可以很好地推广到真实的图像，实现高质量的形状和材料估计，以及基于图像的重新照明。代码、模型和数据将公开发布。
We present a novel physically-motivated deep network for joint shape and material estimation, as well as relighting under novel illumination conditions, using a single image captured by a mobile phone camera. Our physically-based modeling leverages a deep cascaded architecture trained on a large-scale synthetic dataset that consists of complex shapes with microfacet SVBRDF. In contrast to prior works that train rendering layers subsequent to inverse rendering, we propose deep feature sharing and joint training that transfer insights across both tasks, to achieve significant improvements in both reconstruction and relighting. We demonstrate in extensive qualitative and quantitative experiments that our network generalizes very well to real images, achieving high-quality shape and material estimation, as well as image-based relighting. Code, models and data will be publicly released.