Depth-Aware Mirror Segmentation
Depth-Aware Mirror Segmentation
复制标题
DOI：
10.1109/cvpr46437.2021.00306
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Haiyang Mei;Bo Dong;Wen Dong;P. Peers;Xin Yang;Qiang Zhang;Xiaopeng Wei
中科院分区：
文献类型：
--
作者：
Haiyang Mei;Bo Dong;Wen Dong;P. Peers;Xin Yang;Qiang Zhang;Xiaopeng Wei
我们提出了一种新的镜子分割方法，利用基于ToF的相机的深度估计作为额外的线索，消除具有挑战性的情况下，镜子反射和周围场景之间的RGB颜色的对比度或关系是微妙的。一个关键的观察结果是，ToF深度估计并不报告镜面的真实深度，而是返回反射光路的总长度，从而在镜面边界处产生明显的深度不连续性。为了在镜像分割中利用深度信息，我们首先构建了一个大规模的RGB-D镜像分割数据集，随后我们使用它来训练一个新的深度感知镜像分割框架。我们的镜像分割框架首先基于颜色和深度不连续性和相关性定位镜像。接下来，我们的模型通过考虑颜色和深度信息的上下文对比来进一步细化镜像边界。我们广泛地验证了我们的深度感知镜像分割方法，并证明我们的模型优于最先进的基于RGB和RGB-D的镜像分割方法。实验结果还表明，深度是一个强大的提示镜像分割。
We present a novel mirror segmentation method that leverages depth estimates from ToF-based cameras as an additional cue to disambiguate challenging cases where the contrast or relation in RGB colors between the mirror reflection and the surrounding scene is subtle. A key observation is that ToF depth estimates do not report the true depth of the mirror surface, but instead return the total length of the reflected light paths, thereby creating obvious depth dis-continuities at the mirror boundaries. To exploit depth information in mirror segmentation, we first construct a large-scale RGB-D mirror segmentation dataset, which we subsequently employ to train a novel depth-aware mirror segmentation framework. Our mirror segmentation framework first locates the mirrors based on color and depth discontinuities and correlations. Next, our model further refines the mirror boundaries through contextual contrast taking into account both color and depth information. We extensively validate our depth-aware mirror segmentation method and demonstrate that our model outperforms state-of-the-art RGB and RGB-D based methods for mirror segmentation. Experimental results also show that depth is a powerful cue for mirror segmentation.