Learning to reconstruct shape and spatially-varying reflectance from a single image
Learning to reconstruct shape and spatially-varying reflectance from a single image
复制标题
DOI：
10.1145/3272127.3275055
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Zhengqin Li;Zexiang Xu;R. Ramamoorthi;Kalyan Sunkavalli;Manmohan Chandraker
中科院分区：
文献类型：
--
作者：
Zhengqin Li;Zexiang Xu;R. Ramamoorthi;Kalyan Sunkavalli;Manmohan Chandraker
从图像中重建形状和反射率属性是一个高度受限的问题，以前通过使用专门的硬件来捕获校准数据或通过假设已知（或高度受限）的形状或反射率来解决。相比之下，我们证明了我们可以从未知环境照明和闪光灯组合下捕获的单个RGB图像中恢复非兰伯氏，空间变化的brdf和属于任意形状类的复杂几何形状。我们通过训练一个深度神经网络来回归图像的形状和反射率来实现这一点。我们的网络能够解决这个问题是因为三个新颖的贡献：首先，我们建立了一个大规模的程序生成的形状和现实世界的复杂svbrdf数据集，它很好地接近了现实世界的外观。其次，单幅图像的反向渲染需要在多个尺度上进行推理，我们提出了一个级联网络结构，允许以一种易于处理的方式进行推理。最后，我们结合了一个网络内渲染层，通过处理对现实世界场景很重要的全局照明效果来帮助重建任务。总之，这些贡献使我们能够以整体的方式解决整个逆向渲染问题，并在合成和真实数据上产生最先进的结果。
Reconstructing shape and reflectance properties from images is a highly under-constrained problem, and has previously been addressed by using specialized hardware to capture calibrated data or by assuming known (or highly constrained) shape or reflectance. In contrast, we demonstrate that we can recover non-Lambertian, spatially-varying BRDFs and complex geometry belonging to any arbitrary shape class, from a single RGB image captured under a combination of unknown environment illumination and flash lighting. We achieve this by training a deep neural network to regress shape and reflectance from the image. Our network is able to address this problem because of three novel contributions: first, we build a large-scale dataset of procedurally generated shapes and real-world complex SVBRDFs that approximate real world appearance well. Second, single image inverse rendering requires reasoning at multiple scales, and we propose a cascade network structure that allows this in a tractable manner. Finally, we incorporate an in-network rendering layer that aids the reconstruction task by handling global illumination effects that are important for real-world scenes. Together, these contributions allow us to tackle the entire inverse rendering problem in a holistic manner and produce state-of-the-art results on both synthetic and real data.