CI-New: Collaborative Research: Federated Data Set Infrastructure for Recognition Problems in Computer Vision

过去十年来，对图像和视频数据集的广泛访问是计算机视觉识别问题取得很大进展的原因。 These common benchmarks have played a leading role in transforming recognition research from a black art into an experimental science. 然而，进展却停滞不前。尽管数据集不断增长，但它们是孤立开发和注释的：例如，体育活动的集合、图像中的一组对象等。这些孤立的数据集受到任务和领域特定偏差的影响，并且它们之间的知识转移极其有限。 该项目正在研究和建立一个原型架构，通过为跨数据集的实体、事件和注释建立公共命名空间，联合各种识别问题和模式。 该项目还为原型联合数据集架构建立一个门户网站，并将两个现有的识别数据集链接到原型架构中。由此产生的联合结构确实大于各个部分的总和，并且可以支持计算机视觉社区和其他相关领域以前不可能实现的新研究。作为该联合架构的第一个测试场景，该项目正在研究和构建一个新的图像和视频联合数据集，并用各种形式的相关文本进行注释。 图像和视频内容注释跨越空间和时间维度，而反映所描绘内容的文本注释范围从完整的自由形式自然语言描述，到更有针对性的短语和引用表达，再到单独的关键字列表。 该数据集的构建是为了通过提供带有相关研究竞赛的新的多模式注释数据集来促进和加强视觉和语言社区之间的合作。

Broad access to image and video datasets has been responsible for much of the progress in computer vision recognition problems over the last decade. These common benchmarks have played a leading role in transforming recognition research from a black art into an experimental science. Progress, however, has stagnated; although datasets continue to grow, they are developed and annotated in isolation: e.g., a collection of sporting activities, a set of objects in images, etc. These isolated datasets suffer from task and domain-specific bias, and knowledge transfer across them is extremely limited. This project is investigating and establishing a prototype architecture that federates across various recognition problems and modalities, by establishing a common namespace for entities, events and annotations across the datasets. The project is also establishing a web-portal for the prototype federated dataset architecture and linking two existing recognition datasets into the prototype architecture. The resulting federated structure is truly greater than the sum of its parts, and can support new research that was not previously possible for the computer vision community and other related fields.As a first test scenario for this federated architecture, this project is investigating and constructing a new federated dataset of images and video annotated with various forms of associated text. Image and video content annotations span both the spatial and temporal dimensions while textual annotations reflecting depicted content range from complete free-form natural language descriptions, to more targeted phrases and referring expressions, to individual keyword lists. This dataset is being constructed to promote and enhance collaboration efforts between the vision and language communities by providing a new multi-modal annotated dataset with associated research competitions.