Super-Resolution for Monocular Depth Estimation With Multi-Scale Sub-Pixel Convolutions and a Smoothness Constraint
Super-Resolution for Monocular Depth Estimation With Multi-Scale Sub-Pixel Convolutions and a Smoothness Constraint
复制标题
DOI：
10.1109/access.2019.2894651
复制
发表时间：
2019-01
期刊：
影响因子：
3.9
通讯作者：
Shiyu Zhao;Lin Zhang;Ying Shen;Shengjie Zhao;Huijuan Zhang
中科院分区：
文献类型：
--
作者：
Shiyu Zhao;Lin Zhang;Ying Shen;Shengjie Zhao;Huijuan Zhang
从单目图像的深度估计是非常重要的各种视觉任务，如障碍物检测，机器人导航和三维重建。然而，如何获得具有清晰细节和精细分辨率的精确深度图仍然是一个悬而未决的问题。为了解决这一问题，我们利用图像超分辨率的概念和技术进行单目深度估计，并提出了一种新的基于CNN的方法，即$MSCN_{NS}$，它涉及多尺度子像素卷积和邻域平滑约束。具体来说，$MSCN_{NS}$使用具有多尺度融合的子像素卷积来检索具有场景细节的高分辨率深度图。与以往的多尺度融合策略不同，这些多尺度特征来自网络的监督尺度分支。此外，$MSCN_{NS}$结合了邻域平滑正则化项，以确保具有相似特征的空间上更接近的像素将具有接近的深度值。$MSCN_{NS}$的有效性和效率已通过在基准数据集上进行的大量实验得到证实。
Depth estimation from a monocular image is of paramount importance in various vision tasks, such as obstacle detection, robot navigation, and 3D reconstruction. However, how to get an accurate depth map with clear details and a fine resolution remains an unresolved issue. As an attempt to solve this problem, we exploit image super-resolution concepts and techniques for monocular depth estimation and propose a novel CNN-based approach, namely $MSCN_{NS}$ , which involves multi-scale sub-pixel convolutions and a neighborhood smoothness constraint. Specifically, $MSCN_{NS}$ makes use of sub-pixel convolutions with multi-scale fusions to retrieve a high-resolution depth map with fine details of the scene. Different from previous multi-scale fusion strategies, those multi-scale features come from supervised scale branches of the network. Furthermore, $MSCN_{NS}$ incorporates a neighborhood smoothness regularization term to make sure that spatially closer pixels with similar features would have close depth values. The effectiveness and efficiency of $MSCN_{NS}$ have been corroborated through extensive experiments conducted on benchmark datasets.