Unsupervised Discovery and Composition of Object Light Fields
Unsupervised Discovery and Composition of Object Light Fields
复制标题
DOI：
10.48550/arxiv.2205.03923
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Cameron Smith;Hong-Xing Yu;Sergey Zakharov;F. Durand;J. Tenenbaum;Jiajun Wu;V. Sitzmann
中科院分区：
文献类型：
--
作者：
Cameron Smith;Hong-Xing Yu;Sergey Zakharov;F. Durand;J. Tenenbaum;Jiajun Wu;V. Sitzmann
连续和离散的神经场景表示最近已成为 3D 场景理解的强大新范式。最近的工作已经解决了以对象为中心的神经场景表示的无监督发现问题。然而，光线行进的成本很高，而且每个对象表示都必须单独进行光线行进，这会导致辐射场采样不足，从而导致渲染噪声、帧速率差以及训练和渲染期间的内存和时间复杂度较高。在这里，我们建议将以对象为中心的组合场景表示中的对象表示为光场。我们提出了一种新颖的光场合成器模块，可以从一组以对象为中心的光场重建全局光场。我们的方法被称为组合对象光场 (COLF)，能够实现以对象为中心的神经场景表示的无监督学习、标准数据集上最先进的重建和新颖的视图合成性能，以及比现有 3D 方法快几个数量级的渲染和训练速度。
Neural scene representations, both continuous and discrete, have recently emerged as a powerful new paradigm for 3D scene understanding. Recent efforts have tackled unsupervised discovery of object-centric neural scene representations. However, the high cost of ray-marching, exacerbated by the fact that each object representation has to be ray-marched separately, leads to insufficiently sampled radiance fields and thus, noisy renderings, poor framerates, and high memory and time complexity during training and rendering. Here, we propose to represent objects in an object-centric, compositional scene representation as light fields. We propose a novel light field compositor module that enables reconstructing the global light field from a set of object-centric light fields. Dubbed Compositional Object Light Fields (COLF), our method enables unsupervised learning of object-centric neural scene representations, state-of-the-art reconstruction and novel view synthesis performance on standard datasets, and rendering and training speeds at orders of magnitude faster than existing 3D approaches.