DepthCut: improved depth edge estimation using multiple unreliable channels
DepthCut: improved depth edge estimation using multiple unreliable channels
复制标题
DOI：
10.1007/s00371-018-1551-5
复制
发表时间：
2018-09-01
期刊：
影响因子：
3.5
通讯作者：
Mitra, Niloy J.
中科院分区：
文献类型：
--
作者：
Guerrero, Paul;Winnemoller, Holger;Mitra, Niloy J.
在场景理解的背景下，存在多种方法来估计来自单色或立体图像的不同信息通道，包括视差、深度和法线。尽管近年来这些任务取得了一些进展，但估计的信息通常不精确，特别是在深度不连续或折痕附近。然而，研究表明，正是这种深度边缘承载了形状感知的关键线索，并在基于深度的分割或前景选择等任务中发挥着重要作用。不幸的是，当前提取的通道经常携带冲突信号，使得后续应用程序很难有效地使用它们。在本文中，我们重点关注通过联合分析此类不可靠的信息通道来获得高精度深度边缘（即深度轮廓和折痕）的问题。我们提出了 DepthCut，一种数据驱动的通道融合，使用在已知深度的大型数据集上训练的卷积神经网络。生成的深度边缘可用于分割，将场景分解为具有相对平坦深度的深度层，或者通过限制其梯度以与这些边缘一致来提高深度边缘附近的深度估计的准确性。定量地，我们与 18 种基线变体进行比较，并证明与数据无关的通道融合相比，我们的深度边缘可提高分割性能，并改进深度边缘附近的深度估计。定性地，我们证明深度边缘可以带来优异的分割和深度排序。 （将提供代码和数据集。）。
In the context of scene understanding, a variety of methods exists to estimate different information channels from mono or stereo images, including disparity, depth, and normals. Although several advances have been reported in the recent years for these tasks, the estimated information is often imprecise particularly near depth discontinuities or creases. Studies have however shown that precisely such depth edges carry critical cues for the perception of shape, and play important roles in tasks like depth-based segmentation or foreground selection. Unfortunately, the currently extracted channels often carry conflicting signals, making it difficult for subsequent applications to effectively use them. In this paper, we focus on the problem of obtaining high-precision depth edges (i.e., depth contours and creases) by jointly analyzing such unreliable information channels. We propose DepthCut, a data-driven fusion of the channels using a convolutional neural network trained on a large dataset with known depth. The resulting depth edges can be used for segmentation, decomposing a scene into depth layers with relatively flat depth, or improving the accuracy of the depth estimate near depth edges by constraining its gradients to agree with these edges. Quantitatively, we compare against 18 variants of baselines and demonstrate that our depth edges result in an improved segmentation performance and an improved depth estimate near depth edges compared to data-agnostic channel fusion. Qualitatively, we demonstrate that the depth edges result in superior segmentation and depth orderings. (Code and datasets will be made available.).