Neural Subspaces for Light Fields
Neural Subspaces for Light Fields
复制标题
DOI：
10.1109/tvcg.2022.3224674
复制
发表时间：
2022-12
影响因子：
5.2
通讯作者：
Brandon Yushan Feng;Amitabh Varshney
中科院分区：
文献类型：
--
作者：
Brandon Yushan Feng;Amitabh Varshney
我们引入了一个框架，通过神经子空间这一新颖概念对光场内容进行紧凑表示。虽然最近提出的神经光场表示通过将光场编码到单个神经网络中取得了很好的压缩效果，但这种统一设计对于光场中呈现的复合结构并非最优。此外，将光场的每个部分都编码到一个网络中，对于需要快速传输和解码的应用来说并不理想。我们认识到这个问题与子空间学习有关。我们提出了一种方法，该方法使用几个小型神经网络，专门用于学习特定光场片段的神经子空间。此外，我们在这些小型网络之间提出了一种自适应权重共享策略，提高了参数效率。实际上，这种策略能够通过利用神经网络的分层结构，以一种协同的方式追踪相邻神经子空间之间的相似性。进一步地，我们开发了一种软分类技术来提高神经表示的颜色预测准确性。我们的实验结果表明，在各种光场场景下，我们的方法比之前的方法能更好地重建光场。我们还进一步展示了它在对具有不规则视点布局和动态场景内容的光场进行编码方面的成功应用。
We introduce a framework for compactly representing light field content with the novel concept of neural subspaces. While the recently proposed neural light field representation achieves great compression results by encoding a light field into a single neural network, the unified design is not optimized for the composite structures exhibited in light fields. Moreover, encoding every part of the light field into one network is not ideal for applications that require rapid transmission and decoding. We recognize this problem's connection to subspace learning. We present a method that uses several small neural networks, specializing in learning the neural subspace for a particular light field segment. Moreover, we propose an adaptive weight sharing strategy among those small networks, improving parameter efficiency. In effect, this strategy enables a concerted way to track the similarity among nearby neural subspaces by leveraging the layered structure of neural networks. Furthermore, we develop a soft-classification technique to enhance the color prediction accuracy of neural representations. Our experimental results show that our method better reconstructs the light field than previous methods on various light field scenes. We further demonstrate its successful deployment on encoding light fields with irregular viewpoint layout and dynamic scene content.