Robust super-resolution depth imaging via a multi-feature fusion deep network
Robust super-resolution depth imaging via a multi-feature fusion deep network
复制标题
DOI：
10.1364/oe.415563
复制
发表时间：
2021-04-12
期刊：
影响因子：
3.8
通讯作者：
Leach, Jonathan
中科院分区：
文献类型：
--
作者：
Ruget, Alice;McLaughlin, Stephen;Leach, Jonathan
使用深度成像的应用数量正在迅速增加，例如自动驾驶自动驾驶汽车和智能手机相机上的自动对焦辅助。经由单光子敏感探测器（SPAD）阵列的光探测和测距（LIDAR）是一种新兴技术，其使得能够以高帧速率获取深度图像。然而，与传统相机记录的强度图像相比，该技术的空间分辨率通常较低。为了提高SPAD相机深度图像的原生分辨率，我们开发了一个深度网络，该网络利用了可以从相机直方图数据中提取的多个特征。该网络被设计用于以双模式操作的SPAD相机，使得其以高帧速率捕获交替的低分辨率深度和高分辨率强度图像，因此该系统不需要任何附加传感器来提供强度图像。然后，网络使用强度图像和从下采样直方图中提取的多个特征来指导深度的上采样。我们的网络提供了显着的图像分辨率增强和图像去噪在广泛的信噪比和光子水平。此外，我们表明，该网络可以应用到其他数据类型的SPAD数据，证明了算法的通用性。由The Optical Society根据知识共享署名4.0许可证条款发布。
The number of applications that use depth imaging is increasing rapidly, e.g. selfdriving autonomous vehicles and auto-focus assist on smartphone cameras. Light detection and ranging (LIDAR) via single-photon sensitive detector (SPAD) arrays is an emerging technology that enables the acquisition of depth images at high frame rates. However, the spatial resolution of this technology is typically low in comparison to the intensity images recorded by conventional cameras. To increase the native resolution of depth images from a SPAD camera, we develop a deep network built to take advantage of the multiple features that can be extracted from a camera's histogram data. The network is designed for a SPAD camera operating in a dual-mode such that it captures alternate low resolution depth and high resolution intensity images at high frame rates, thus the system does not require any additional sensor to provide intensity images. The network then uses the intensity images and multiple features extracted from down-sampled histograms to guide the up-sampling of the depth. Our network provides significant image resolution enhancement and image denoising across a wide range of signal-to-noise ratios and photon levels. Additionally, we show that the network can be applied to other data types of SPAD data, demonstrating the generality of the algorithm. Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License.