MaterialGAN: reflectance capture using a generative SVBRDF model
MaterialGAN: reflectance capture using a generative SVBRDF model
复制标题
MaterialGAN：使用生成 SVBRDF 模型进行反射率捕获
DOI：
10.1145/3414685.3417779
复制
发表时间：
2020
影响因子：
6.2
通讯作者：
Zhao, Shuang
中科院分区：
文献类型：
--
作者：
Guo, Yu;Smith, Cameron;Hašan, Miloš;Sunkavalli, Kalyan;Zhao, Shuang
我们解决了从一小组图像测量重建空间变化brdf的问题。这基本上是一个缺乏约束的问题，以前的工作依赖于使用各种正则化先验或捕获许多图像来产生可信的结果。在这项工作中，我们提出了MaterialGAN，一个基于StyleGAN2的深度生成卷积网络，用于合成逼真的SVBRDF参数图。我们展示了MaterialGAN可以在一个反向渲染框架中作为一个强大的材料：我们优化了它的潜在表示，以生成与渲染时捕获的图像外观相匹配的材料地图。我们在使用手持移动电话从闪光灯照明下捕获的图像中重建svbrdf的任务上演示了该框架。我们的方法成功地生成了可信的材料图，准确地再现了目标图像，并且在合成和真实数据的评估中优于以前最先进的材料捕获方法。此外，我们基于gan的潜在空间允许高级语义材料编辑操作，如生成材料变化和材料变形。
We address the problem of reconstructing spatially-varying BRDFs from a small set of image measurements. This is a fundamentally under-constrained problem, and previous work has relied on using various regularization priors or on capturing many images to produce plausible results. In this work, we present MaterialGAN, a deep generative convolutional network based on StyleGAN2, trained to synthesize realistic SVBRDF parameter maps. We show that MaterialGAN can be used as a powerful material prior in an inverse rendering framework: we optimize in its latent representation to generate material maps that match the appearance of the captured images when rendered. We demonstrate this framework on the task of reconstructing SVBRDFs from images captured under flash illumination using a hand-held mobile phone. Our method succeeds in producing plausible material maps that accurately reproduce the target images, and outperforms previous state-of-the-art material capture methods in evaluations on both synthetic and real data. Furthermore, our GAN-based latent space allows for high-level semantic material editing operations such as generating material variations and material morphing.