Self-Supervised Single-Image Depth Estimation From Focus and Defocus Clues
Self-Supervised Single-Image Depth Estimation From Focus and Defocus Clues
复制标题
DOI：
10.1109/lra.2021.3092258
复制
发表时间：
2021-10
影响因子：
5.2
通讯作者：
Yawen Lu;Garrett Milliron;John Slagter;G. Lu
中科院分区：
文献类型：
--
作者：
Yawen Lu;Garrett Milliron;John Slagter;G. Lu
自监督深度估计最近表现出有前途的性能相比，具有挑战性的室内场景的监督方法。然而，大多数的努力主要集中在利用光度和几何一致性，通过前向图像变形和后向图像变形，基于单目视频或立体图像对。散焦模糊对深度估计的影响被忽略，导致对失焦的物体和场景的性能有限。在这项工作中，我们提出了第一个框架，同时从一个单一的图像和图像焦点堆栈使用深度从散焦和深度从焦点算法的深度估计。所提出的网络能够从单个图像的模糊中包含的信息学习最佳深度映射，生成模拟图像焦点堆栈和全焦点图像，并从图像焦点堆栈训练深度估计器。除了在合成的NYUv2数据集和真实的DSLR数据集上验证我们的方法外，我们还使用DSLR相机收集了我们自己的数据集并进一步验证。实验表明，我们的系统在精度上超过了最先进的监督深度估计方法4%，并且在合成的NYUv2数据集上没有直接监督的方法中取得了优异的性能。很少有人探索过
Self-supervised depth estimation has recently demonstrated promising performance compared to the supervised methods on challenging indoor scenes. However, the majority of efforts mainly focus on exploiting photometric and geometric consistency via forward image warping and backward image warping, based on monocular videos or stereo image pairs. The influence of defocus blur to depth estimation is neglected, resulting in a limited performance for objects and scenes in out of focus. In this work, we propose the first framework for simultaneous depth estimation from a single image and image focal stacks using depth-from-defocus and depth-from-focus algorithms. The proposed network is able to learn optimal depth mapping from the information contained in the blur of a single image, generate a simulated image focal stack and all-in-focus image, and train a depth estimator from an image focal stack. In addition to the validation of our method on both synthetic NYUv2 dataset and real DSLR dataset, we also collect our own dataset using a DSLR camera and further verify on it. Experiments demonstrate that our system surpasses the state-of-the-art supervised depth estimation method over 4% in accuracy and achieves superb performance among the methods without direct supervision on the synthesized NYUv2 dataset, which has been rarely explored.