Comparison of super-resolution and noise reduction for passive single-photon imaging
Comparison of super-resolution and noise reduction for passive single-photon imaging
复制标题
DOI：
10.1117/1.jei.31.3.033042
复制
发表时间：
2022-05
影响因子：
1.1
通讯作者：
Martin Laurenzis;T. Seets;E. Bacher;A. Ingle;A. Velten
中科院分区：
文献类型：
--
作者：
Martin Laurenzis;T. Seets;E. Bacher;A. Ingle;A. Velten
抽象的。单光子敏感图像传感器最近在被动成像应用中得到普及，其中目标是在具有挑战性的照明条件和场景运动的存在下捕获不同场景点的光子通量（亮度）值。最近的工作表明，使用单光子雪崩二极管相机捕获的单光子时间戳信息的高速突发可以用于估计和校正场景运动，从而提高信噪比并减少运动模糊伪影。我们在用于降噪、运动补偿和单光子时间戳帧的上采样的处理流水线中执行各种设计选择的比较。我们考虑将各种像素降噪技术与最先进的深度神经网络升级算法相结合，以超分辨由单光子时间戳数据形成的强度图像。我们探讨了在不同的运动内容的各种场景中的运动模糊和信号噪声的交易空间。使用硬件原型捕获的真实的数据，我们以高达65.8 kHz的帧速率（传感器的本地采样率）实现了超分辨率重建，并捕获了快速移动物体的视频。最好的重建与运动补偿的方法，它实现了快速移动的刚性物体的结构相似性（SSIM）约为0.67。我们能够重建亚像素分辨率。这些结果表明，我们的运动补偿相比，不超过0.5的SSIM的其他方法的相对优越性。
Abstract. Single-photon sensitive image sensors have recently gained popularity in passive imaging applications where the goal is to capture photon flux (brightness) values of different scene points in the presence of challenging lighting conditions and scene motion. Recent work has shown that high-speed bursts of single-photon timestamp information captured using a single-photon avalanche diode camera can be used to estimate and correct for scene motion thereby improving signal-to-noise ratio and reducing motion blur artifacts. We perform a comparison of various design choices in the processing pipeline used for noise reduction, motion compensation, and upsampling of single-photon timestamp frames. We consider various pixelwise noise reduction techniques in combination with state-of-the-art deep neural network upscaling algorithms to super-resolve intensity images formed with single-photon timestamp data. We explore the trade space of motion blur and signal noise in various scenes with different motion content. Using real data captured with a hardware prototype, we achieved super-resolution reconstruction at frame rates up to 65.8 kHz (native sampling rate of the sensor) and captured videos of fast-moving objects. The best reconstruction is obtained with the motion compensation approach, which achieves a structural similarity (SSIM) of about 0.67 for fast-moving rigid objects. We are able to reconstruct subpixel resolution. These results show the relative superiority of our motion compensation compared to other approaches that do not exceed an SSIM of 0.5.