Neural Distortion Fields for Spatial Calibration of Wide Field-of-View Near-Eye Displays
Neural Distortion Fields for Spatial Calibration of Wide Field-of-View Near-Eye Displays
复制标题
DOI：
10.1364/oe.472288
复制
发表时间：
2022-10
期刊：
影响因子：
3.8
通讯作者：
Yuichi Hiroi;Kiyosato Someya;Yuta Itoh
中科院分区：
文献类型：
--
作者：
Yuichi Hiroi;Kiyosato Someya;Yuta Itoh
我们提出了一种用于具有复杂图像失真的宽视场（FoV）近眼显示器（NED）的空间校准方法。 NED 中的图像扭曲会破坏虚拟对象的真实性并导致疾病。为了在 NED 中获得无失真图像，有必要在视点和显示图像之间建立逐像素对应关系。设计紧凑且宽视场 NED 需要复杂的光学设计。在此类设计中，显示的图像会受到注视相关的非线性几何失真的影响，这些显式几何模型可能难以表示或需要大量计算来优化。为了解决这些问题，我们提出了神经扭曲场（NDF），这是一种全连接的深度神经网络，它隐式地表示空间中复杂扭曲的显示表面。 NDF 将空间位置和注视方向作为输入，并输出在输入注视方向上感知的显示像素坐标及其强度。我们通过从视点查询射线上的点并计算加权和以将输出显示坐标投影到图像中，从新的视点合成畸变图。实验表明，NDF 仅使用 8 个训练视点即可校准具有 90° FoV 的增强现实 NED，中值误差约为 3.23 像素（5.8 弧分）。此外，我们确认 NDF 的校准比非线性多项式拟合更准确，尤其是在 FoV 中心附近。
We propose a spatial calibration method for wide field-of-view (FoV) near-eye displays (NEDs) with complex image distortions. Image distortions in NEDs can destroy the reality of the virtual object and cause sickness. To achieve distortion-free images in NEDs, it is necessary to establish a pixel-by-pixel correspondence between the viewpoint and the displayed image. Designing compact and wide-FoV NEDs requires complex optical designs. In such designs, the displayed images are subject to gaze-contingent, non-linear geometric distortions, which explicit geometric models can be difficult to represent or computationally intensive to optimize. To solve these problems, we propose neural distortion field (NDF), a fully-connected deep neural network that implicitly represents display surfaces complexly distorted in spaces. NDF takes spatial position and gaze direction as input and outputs the display pixel coordinate and its intensity as perceived in the input gaze direction. We synthesize the distortion map from a novel viewpoint by querying points on the ray from the viewpoint and computing a weighted sum to project output display coordinates into an image. Experiments showed that NDF calibrates an augmented reality NED with 90° FoV with about 3.23 pixel (5.8 arcmin) median error using only 8 training viewpoints. Additionally, we confirmed that NDF calibrates more accurately than the non-linear polynomial fitting, especially around the center of the FoV.