CI-New: Collaborative Research: Federated Data Set Infrastructure for Recognition Problems in Computer Vision

在过去十年中，对图像和视频数据集的广泛访问是计算机视觉识别问题取得很大进展的原因。这些共同的基准在将识别研究从黑色艺术转变为实验科学方面发挥了主导作用。然而，进展停滞不前；虽然数据集继续增长，但它们是独立开发和注释的：例如，体育活动的集合、图像中的一组对象等。这些孤立的数据集受到任务和领域特定偏差的影响，并且它们之间的知识转移非常有限。该项目通过为数据集上的实体、事件和注释建立一个通用的命名空间，研究并建立了一个原型架构，该架构可以跨各种识别问题和模式进行联合。该项目还为原型联邦数据集架构建立了一个门户网站，并将两个现有的识别数据集连接到原型架构中。由此产生的联合结构确实比其各部分的总和更大，并且可以支持计算机视觉社区和其他相关领域以前不可能进行的新研究。作为这个联邦体系结构的第一个测试场景，这个项目正在研究和构建一个新的联邦图像和视频数据集，这些数据集使用各种形式的相关文本进行注释。图像和视频内容注释跨越空间和时间维度，而反映所描述内容的文本注释范围从完全自由形式的自然语言描述到更有针对性的短语和引用表达式，再到单个关键字列表。这个数据集的构建是为了通过提供一个新的多模态注释数据集和相关的研究竞赛来促进和加强视觉和语言社区之间的协作努力。

Broad access to image and video datasets has been responsible for much of the progress in computer vision recognition problems over the last decade. These common benchmarks have played a leading role in transforming recognition research from a black art into an experimental science. Progress, however, has stagnated; although datasets continue to grow, they are developed and annotated in isolation: e.g., a collection of sporting activities, a set of objects in images, etc. These isolated datasets suffer from task and domain-specific bias, and knowledge transfer across them is extremely limited. This project is investigating and establishing a prototype architecture that federates across various recognition problems and modalities, by establishing a common namespace for entities, events and annotations across the datasets. The project is also establishing a web-portal for the prototype federated dataset architecture and linking two existing recognition datasets into the prototype architecture. The resulting federated structure is truly greater than the sum of its parts, and can support new research that was not previously possible for the computer vision community and other related fields.As a first test scenario for this federated architecture, this project is investigating and constructing a new federated dataset of images and video annotated with various forms of associated text. Image and video content annotations span both the spatial and temporal dimensions while textual annotations reflecting depicted content range from complete free-form natural language descriptions, to more targeted phrases and referring expressions, to individual keyword lists. This dataset is being constructed to promote and enhance collaboration efforts between the vision and language communities by providing a new multi-modal annotated dataset with associated research competitions.