Neural 3D Holography: Learning Accurate Wave Propagation Models for 3D Holographic Virtual and Augmented Reality Displays
Neural 3D Holography: Learning Accurate Wave Propagation Models for 3D Holographic Virtual and Augmented Reality Displays
复制标题
DOI：
10.1145/3478513.3480542
复制
发表时间：
2021-12-01
影响因子：
6.2
通讯作者：
Wetzstein, Gordon
中科院分区：
文献类型：
--
作者：
Choi, Suyeon;Gopakumar, Manu;Wetzstein, Gordon
全息近眼显示器为虚拟和增强现实（VR/AR）系统提供了前所未有的功能。然而，当前全息显示器的图像质量受到用于模拟物理光学的波传播模型的限制。我们提出了一个神经网络参数化的平面到多平面的波传播模型，它缩小了物理和模拟之间的差距。我们的模型是使用相机反馈自动训练的，它在2D平面到平面的设置中大大优于相关技术。此外，它是第一个自然扩展到3D设置的网络参数化模型，使用复值波场的新相位正则化策略实现高质量的3D计算机生成全息。通过对VR和光学透明AR显示原型进行广泛的实验评估，证明了我们方法的有效性。
Holographic near-eye displays promise unprecedented capabilities for virtual and augmented reality (VR/AR) systems. The image quality achieved by current holographic displays, however, is limited by the wave propagation models used to simulate the physical optics. We propose a neural network-parameterized plane-to-multiplane wave propagation model that closes the gap between physics and simulation. Our model is automatically trained using camera feedback and it outperforms related techniques in 2D plane-to-plane settings by a large margin. Moreover, it is the first network-parameterized model to naturally extend to 3D settings, enabling high-quality 3D computer-generated holography using a novel phase regularization strategy of the complex-valued wave field. The efficacy of our approach is demonstrated through extensive experimental evaluation with both VR and optical see-through AR display prototypes.