The Open Images Dataset V4 Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale
The Open Images Dataset V4 Unified Image Classification, Object Detection, and Visual Relationship Detection at Scale
复制标题
DOI：
10.1007/s11263-020-01316-z
复制
发表时间：
2020-03-13
影响因子：
19.5
通讯作者：
Ferrari, Vittorio
中科院分区：
文献类型：
--
作者：
Kuznetsova, Alina;Rom, Hassan;Ferrari, Vittorio
我们提出了Open Images V4，这是一个包含920万张图像的数据集，具有用于图像分类，对象检测和视觉关系检测的统一注释。这些图片有一个知识共享署名许可证，允许分享和改编材料，它们是从Flickr收集的，没有预先定义的类名或标签列表，从而产生自然的类统计数据，避免了最初的设计偏见。Open Images V4提供了跨多个维度的大规模：3010万个图像级标签用于1980万个概念，1540万个边界框用于600个对象类，以及3750万个涉及57个类的视觉关系注释。特别是对于对象检测，我们提供的边界框比下一个最大的数据集（190万张图像上的1540万个框）多15倍。图像通常显示具有多个对象的复杂场景（平均每个图像8个注释对象）。我们标注了它们之间的视觉关系，这支持视觉关系检测，一个新兴的任务，需要结构化推理。我们提供了关于数据集的深入全面的统计数据，我们验证了注释的质量，我们研究了几种现代模型的性能如何随着训练数据量的增加而演变，我们展示了两个应用程序，这些应用程序通过在同一图像中共存多种类型的统一注释而成为可能。我们希望Open Images V4的规模、质量和多样性将促进进一步的研究和创新，甚至超越图像分类、对象检测和视觉关系检测领域。
We present Open Images V4, a dataset of 9.2M images with unified annotations for image classification, object detection and visual relationship detection. The images have a Creative Commons Attribution license that allows to share and adapt the material, and they have been collected from Flickr without a predefined list of class names or tags, leading to natural class statistics and avoiding an initial design bias. Open Images V4 offers large scale across several dimensions: 30.1M image-level labels for 19.8k concepts, 15.4M bounding boxes for 600 object classes, and 375k visual relationship annotations involving 57 classes. For object detection in particular, we provide 15x more bounding boxes than the next largest datasets (15.4M boxes on 1.9M images). The images often show complex scenes with several objects (8 annotated objects per image on average). We annotated visual relationships between them, which support visual relationship detection, an emerging task that requires structured reasoning. We provide in-depth comprehensive statistics about the dataset, we validate the quality of the annotations, we study how the performance of several modern models evolves with increasing amounts of training data, and we demonstrate two applications made possible by having unified annotations of multiple types coexisting in the same images. We hope that the scale, quality, and variety of Open Images V4 will foster further research and innovation even beyond the areas of image classification, object detection, and visual relationship detection.