Why Accuracy is Not Enough: The Need for Consistency in Object Detection
Why Accuracy is Not Enough: The Need for Consistency in Object Detection
复制标题
DOI：
10.1109/mmul.2022.3175239
复制
发表时间：
2022-07
期刊：
影响因子：
3.2
通讯作者：
Caleb Tung;Abhinav Goel;Fischer Bordwell;Nick Eliopoulos;Xiao Hu;Yung-Hsiang Lu;G. Thiruvathukal
中科院分区：
文献类型：
--
作者：
Caleb Tung;Abhinav Goel;Fischer Bordwell;Nick Eliopoulos;Xiao Hu;Yung-Hsiang Lu;G. Thiruvathukal
物体检测器对于许多现代计算机视觉应用至关重要。然而，即使是最先进的物体探测器也不是完美的。在两张看起来与人眼相似的图像上，同一个检测器可以做出不同的预测，因为图像失真很小，比如相机传感器噪声和照明变化。这个问题被称为不一致性。现有的准确性指标没有适当考虑不一致性，在这方面的类似工作只针对人工图像失真的改进。因此，我们提出了一种方法，使用非人工视频帧来衡量对象检测一致性随着时间的推移，跨帧。使用这种方法，我们表明，现代对象检测器的一致性范围从83.2%到97.1%的不同的视频数据集从多个对象跟踪的挑战。我们的结论是，应用图像失真校正，如WEBP图像压缩和反锐化掩码可以提高一致性高达5.1%，而不会损失准确性。
Object detectors are vital to many modern computer vision applications. However, even state-of-the-art object detectors are not perfect. On two images that look similar to human eyes, the same detector can make different predictions because of small image distortions like camera sensor noise and lighting changes. This problem is called inconsistency. Existing accuracy metrics do not properly account for inconsistency, and similar work in this area only targets improvements on artificial image distortions. Therefore, we propose a method to use nonartificial video frames to measure object detection consistency over time, across frames. Using this method, we show that the consistency of modern object detectors ranges from 83.2% to 97.1% on different video datasets from the multiple object tracking challenge. We conclude by showing that applying image distortion corrections such as WEBP Image Compression and Unsharp Masking can improve consistency by as much as 5.1%, with no loss in accuracy.