A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit
A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit
复制标题
DOI：
10.3390/electronics10030279
复制
发表时间：
2021-02-01
期刊：
影响因子：
2.9
通讯作者：
da Silva, Eduardo A. B.
中科院分区：
文献类型：
--
作者：
Padilla, Rafael;Passos, Wesley L.;da Silva, Eduardo A. B.
最近在竞赛和挑战中监督对象检测的出色结果通常与特定的指标和数据集相关。在不同情况下应用的这些方法的评价增加了对注释数据集的需求。注释工具以不同的格式表示对象的位置和大小，导致在表示上缺乏共识。这样的场景通常使对象检测方法的比较复杂化。这项工作沿着以下路线阐述了这个问题：（i）它提供了一个概述的最相关的评估方法中使用的目标检测比赛，突出其特点，差异和优势;（ii）它检查了最常用的注释格式，显示不同的实现可能会影响评估结果;和（iii）它提供了一个新的开源工具包，支持不同的注释格式和15个性能指标，使研究人员能够轻松评估他们的检测算法在大多数已知数据集中的性能。此外，这项工作提出了一个新的度量，也包括在工具包中，用于评估视频中的对象检测，该视频基于地面实况和检测到的边界框之间的时空重叠。
Recent outstanding results of supervised object detection in competitions and challenges are often associated with specific metrics and datasets. The evaluation of such methods applied in different contexts have increased the demand for annotated datasets. Annotation tools represent the location and size of objects in distinct formats, leading to a lack of consensus on the representation. Such a scenario often complicates the comparison of object detection methods. This work alleviates this problem along the following lines: (i) It provides an overview of the most relevant evaluation methods used in object detection competitions, highlighting their peculiarities, differences, and advantages; (ii) it examines the most used annotation formats, showing how different implementations may influence the assessment results; and (iii) it provides a novel open-source toolkit supporting different annotation formats and 15 performance metrics, making it easy for researchers to evaluate the performance of their detection algorithms in most known datasets. In addition, this work proposes a new metric, also included in the toolkit, for evaluating object detection in videos that is based on the spatio-temporal overlap between the ground-truth and detected bounding boxes.