CI-P:Collaborative Research: Visual entailment data set and challenge for the language and vision communities

CI-P：协作研究：视觉蕴含数据集以及语言和视觉社区面临的挑战

• 批准号: 1205354
• 负责人: Tamara Berg
• 金额: $ 5.72万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-07-01 至 2014-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1205354&HistoricalAwards=false
• 关键词: CI; Collaborative; Research; Visual; entailment

Vision and language provide fundamental means to interpret, learn, and communicate about the world around us. A primary goal of computer vision and natural language processing research is therefore to automatically uncover and analyze the information that images and video, or text and speech, convey about the world. Both communities are concerned with tasks that require increasingly deeper understanding, including the ability to reason with and draw inferences from this information. Since vision and language are complementary modalities, there is now also an increasing amount of work at the interface of both fields. However, progress in multimodal analysis requires a tighter collaboration between the two communities, since each currently relies on its own set of techniques, datasets and evaluation criteria. This community planning grant explores the need for, feasibility, and usefulness of a "visual entailment" corpus and associated visual entailment recognition task. In natural language, entailment recognition is the problem of determining whether a particular statement can be inferred from a text document. This project explores a novel related problem - visual entailment - where the goal is to determine whether a statement in natural language can be inferred from an image or video. The outcomes of the project include a novel dataset and prototype research challenge, as well as increased collaboration between the vision and language communities.

视觉和语言提供了解释、学习和交流我们周围世界的基本手段。因此，计算机视觉和自然语言处理研究的主要目标是自动发现和分析图像和视频或文本和语音传达的关于世界的信息。 这两个社区都关注需要越来越深入理解的任务，包括从这些信息中推理和推断的能力。 由于视觉和语言是互补的模式，现在在这两个领域的界面上也有越来越多的工作。然而，要在多模式分析方面取得进展，就需要这两个群体之间加强合作，因为目前每个群体都依赖自己的一套技术、数据集和评价标准。这个社区规划补助金探讨了“视觉蕴涵”语料库和相关的视觉蕴涵识别任务的必要性，可行性和实用性。在自然语言中，蕴涵识别是确定是否可以从文本文档中推断出特定语句的问题。这个项目探讨了一个新的相关问题-视觉蕴涵-其目标是确定是否可以从图像或视频中推断出自然语言中的语句。 该项目的成果包括一个新的数据集和原型研究挑战，以及视觉和语言社区之间的合作。