Acquiring a Dynamic Light Field through a Single-Shot Coded Image-猫眼课题宝

DOI：

10.1109/cvpr52688.2022.01921

发表时间：

2022-04

期刊：

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Ryoya Mizuno;Keita Takahashi;Michitaka Yoshida;Chihiro Tsutake;T. Fujii;H. Nagahara

我们提出了一种通过单镜头编码图像（二维测量）压缩获取动态光场（5维体积）的方法。我们设计了一个成像模型，该模型在单个曝光时间内同步应用光圈编码和逐像素曝光编码。这种编码方案使我们能够有效地将原始信息嵌入到单个观察图像中。然后将观察到的图像馈送到卷积神经网络（CNN）进行光场重建，该网络与相机侧编码模式共同训练。我们还开发了一个硬件原型来捕捉真实的3d场景。我们成功地从单个观测图像中获得了4个时间子帧（总共100个视图）的5x5视点的动态光场。随着时间的推移，重复捕获和重建过程，我们可以获得4倍于相机帧速率的动态光场。据我们所知，我们的方法是第一个在压缩光场采集中实现比相机本身更精细的时间分辨率的方法。我们的软件可从我们的项目网页11https://www.fujii.nuee.nagoya-u.ac.jp/Research/CompCam2

We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5x5 viewpoints over 4 temporal sub-frames (100 views in total)from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4x the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage.11https://www.fujii.nuee.nagoya-u.ac.jp/Research/CompCam2

Acquiring a Dynamic Light Field through a Single-Shot Coded Image

文献摘要

被引文献

相似文献