The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation
复制标题
DOI：
10.1007/s11263-021-01511-6
复制
发表时间：
2021-09-14
影响因子：
19.5
通讯作者：
Cadena, Cesar
中科院分区：
文献类型：
--
作者：
Blum, Hermann;Sarlin, Paul-Edouard;Cadena, Cesar
深度学习在语义分割的准确性方面取得了令人印象深刻的进步。然而，评估不确定性和检测故障的能力对于自动驾驶等安全关键应用至关重要。现有的不确定性估计大多是在简单的任务上进行评估的，目前尚不清楚这些方法是否可以推广到更复杂的情况。我们提出了Fishyscapes，这是在城市驾驶的语义分割的现实世界任务中进行异常检测的第一个公开基准。它评估了对异常物体检测的像素不确定性估计。我们将最先进的方法应用于最新的语义分割模型，并比较了基于softmax置信度、贝叶斯学习、密度估计、图像重新合成以及监督异常检测方法的不确定性估计方法。我们的结果表明，即使在普通情况下，异常检测也远远没有解决，而我们的基准允许测量超越最先进的进展。结果、数据和提交信息可在https://fishyscapes.com/上找到。
Deep learning has enabled impressive progress in the accuracy of semantic segmentation. Yet, the ability to estimate uncertainty and detect failure is key for safety-critical applications like autonomous driving. Existing uncertainty estimates have mostly been evaluated on simple tasks, and it is unclear whether these methods generalize to more complex scenarios. We present Fishyscapes, the first public benchmark for anomaly detection in a real-world task of semantic segmentation for urban driving. It evaluates pixel-wise uncertainty estimates towards the detection of anomalous objects. We adapt state-of-the-art methods to recent semantic segmentation models and compare uncertainty estimation approaches based on softmax confidence, Bayesian learning, density estimation, image resynthesis, as well as supervised anomaly detection methods. Our results show that anomaly detection is far from solved even for ordinary situations, while our benchmark allows measuring advancements beyond the state-of-the-art. Results, data and submission information can be found at https://fishyscapes.com/.