Delving into Light-Dark Semantic Segmentation for Indoor Scenes Understanding
Delving into Light-Dark Semantic Segmentation for Indoor Scenes Understanding
复制标题
DOI：
10.1145/3552482.3556556
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Xiaowen Ying;Bo Lang;Zhihao Zheng;M. Chuah
中科院分区：
文献类型：
--
作者：
Xiaowen Ying;Bo Lang;Zhihao Zheng;M. Chuah
目前最先进的分割模型大多是在光照条件良好的条件下收集的大规模数据集来训练的，因此直接将这些训练好的模型应用到黑暗场景中会导致不理想的性能。在本文中，我们提出了第一个基准数据集和评价方法来研究不同光照条件下室内场景的语义分割问题。我们的数据集，即LDIS，包括从87个不同的室内场景中收集的样本，在光照和弱光条件下都是如此。与已有的工作不同，我们的基准测试提供了一个新的任务设置，即明暗语义分割(LDSS)，它采用了四种不同的评估指标，从多个方面评估了模型的性能。我们进行了广泛的实验和消融研究，以比较不同现有技术和我们的标准化评估方案的有效性。此外，我们提出了一种新的技术，即DepthAux，该技术利用不同光照条件下深度图像的一致性来帮助模型学习统一的、光照不变的表示。我们的实验结果表明，当应用于各种不同的模型时，所提出的DepthAux可以提供一致且显著的改进。我们的数据集和其他资源在我们的项目页面上公开提供：http://mercy.cse.lehigh.edu/LDIS/
State-of-the-art segmentation models are mostly trained with large-scale datasets collected under favorable lighting conditions, and hence directly applying such trained models to dark scenes will result in unsatisfactory performance. In this paper, we present the first benchmark dataset and evaluation methodology to study the problem of semantic segmentation under different lighting conditions for indoor scenes. Our dataset, namely LDIS, consists of samples collected from 87 different indoor scenes under both well-illuminated and low-light conditions. Different from existing work, our benchmark provides a new task setting, namely Light-Dark Semantic Segmentation (LDSS), which adopts four different evaluation metrics that assess the performance of a model from multiple aspects. We perform extensive experiments and ablation studies to compare the effectiveness of different existing techniques with our standardized evaluation protocol. In addition, we propose a new technique, namely DepthAux, that utilizes the consistency of depth images under different lighting conditions to help a model learn a unified and illumination-invariant representation. Our experimental results show that the proposed DepthAux can provide consistent and significant improvements when applied to a variety of different models. Our dataset and other resources are publicly available on our project page: http://mercy.cse.lehigh.edu/LDIS/