Diffeomorphic Neural Surface Parameterization for 3D and Reflectance Acquisition
Diffeomorphic Neural Surface Parameterization for 3D and Reflectance Acquisition
复制标题
DOI：
10.1145/3528233.3530741
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Ziang Cheng;Hongdong Li;R. Hartley;Yinqiang Zheng;Imari Sato
中科院分区：
文献类型：
--
作者：
Ziang Cheng;Hongdong Li;R. Hartley;Yinqiang Zheng;Imari Sato
本文提出了一种简单的方法，解决了未知的和通用的表面材料的物体，由一个自由移动的摄像机成像和自由移动的点光源照明的多视图三维重建的问题。对象可以具有任意（漫反射或镜面反射）和空间变化的表面反射率。我们的解决方案由两个小型神经网络（称为“形状网络”和“BRDF网络”）组成，用于将未知形状和材料映射参数化为规范表面（例如单位球体）上的函数。我们的方法的关键是一个速度场形状表示，驱动规范表面的目标形状通过时间。我们证明了这种参数化可以实现为一个经常性的残差网络，保证是同构和方向保持。我们的方法产生了一个非常干净的配方，可以通过标准梯度下降优化，无需初始化，并与近场和远场光源。合成和真实的实验证明了我们重建的可靠性和准确性，扩展包括新颖的视图合成，重新照明和材料修饰轻松完成。我们的源代码可以在https://github.com/za-cheng/DNS上找到。
This paper proposes a simple method which solves the problem of multi-view 3D reconstruction for objects with unknown and generic surface materials, imaged by a freely moving camera and lit by a freely moving point light source. The object can have arbitrary (diffuse or specular) and spatially-varying surface reflectances. Our solution consists of two small-sized neural networks (dubbed the ‘Shape-Net’ and ‘BRDF-Net’), used to parameterize the unknown shape and material map as functions on a canonical surface (e.g. unit sphere). Key to our method is a velocity field shape representation that drives the canonical surface to target shape through time. We show this parameterization can be implemented as a recurrent residual network that is guaranteed to be diffeomorphic and orientation-preserving. Our method yields an exceptionally clean formulation that can be optimized by standard gradient descent without initialization, and works with both near-field and distant light source. Synthetic and real experiments demonstrate the reliability and accuracy of our reconstructions, with extensions including novel-view-synthesis, relighting and material retouching done with ease. Our source codes are available at https://github.com/za-cheng/DNS.