Scalable neural indoor scene rendering
Scalable neural indoor scene rendering
复制标题
DOI：
10.1145/3528223.3530153
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Xiuchao Wu;Jiamin Xu;Zihan Zhu;H. Bao;Qi-Xing Huang;J. Tompkin;Weiwei Xu
中科院分区：
文献类型：
--
作者：
Xiuchao Wu;Jiamin Xu;Zihan Zhu;H. Bao;Qi-Xing Huang;J. Tompkin;Weiwei Xu
我们提出了一种可扩展的神经场景重建和绘制方法，以支持分布式训练和大型室内场景的交互式绘制。我们的代表是基于瓷砖。通过背景采样策略并行训练瓦片外观，该背景采样策略通过代理全局网格用远处场景信息增强每个瓦片。每个平铺都有两个低容量的MLP：一个用于视图独立外观（漫反射颜色和着色），另一个用于视图相关外观（镜面高光、反射）。我们利用了这样的现象：复杂的与视图相关的场景反射可以归因于表面下方与光源的总光线距离处的虚拟灯光。这使我们能够处理输入场景的稀疏采样，其中反射高光在输入图像中并不总是一致地出现。我们展示了五个场景的交互式自由视点渲染结果，其中一个场景的面积超过100平方米。实验结果表明，我们的方法产生更高质量的渲染比一个单一的大容量MLP和最近的五个神经代理几何体和基于体素的基线方法。我们的代码和数据可以在项目网页https://xchaowu.github.io/papers/scalable-nisr上找到。
We propose a scalable neural scene reconstruction and rendering method to support distributed training and interactive rendering of large indoor scenes. Our representation is based on tiles. Tile appearances are trained in parallel through a background sampling strategy that augments each tile with distant scene information via a proxy global mesh. Each tile has two low-capacity MLPs: one for view-independent appearance (diffuse color and shading) and one for view-dependent appearance (specular highlights, reflections). We leverage the phenomena that complex view-dependent scene reflections can be attributed to virtual lights underneath surfaces at the total ray distance to the source. This lets us handle sparse samplings of the input scene where reflection highlights do not always appear consistently in input images. We show interactive free-viewpoint rendering results from five scenes, one of which covers an area of more than 100 m2. Experimental results show that our method produces higher-quality renderings than a single large-capacity MLP and five recent neural proxy-geometry and voxel-based baseline methods. Our code and data are available at project webpage https://xchaowu.github.io/papers/scalable-nisr.