Grounding Action Descriptions in Videos
Grounding Action Descriptions in Videos
复制标题
DOI：
10.1162/tacl_a_00207
复制
发表时间：
2013-03
影响因子：
10.9
通讯作者：
Michaela Regneri;Marcus Rohrbach;Dominikus Wetzel;Stefan Thater;B. Schiele;Manfred Pinkal
中科院分区：
文献类型：
--
作者：
Michaela Regneri;Marcus Rohrbach;Dominikus Wetzel;Stefan Thater;B. Schiele;Manfred Pinkal
最近的工作表明，将视觉信息集成到基于文本的模型中可以显着改善模型预测，但到目前为止仅使用从静态图像中提取的视觉信息。在本文中，我们考虑从视频中提取的视觉信息中描述动作的基础句子的问题。我们提出了一个通用语料库，将高质量视频与视频中描绘的动作的多种自然语言描述相结合，以及动作描述彼此相似程度的注释。实验结果表明，当与描述所描述动作的视频中的视觉信息相结合时，基于文本的动作之间相似性模型得到了显着改善。
Recent work has shown that the integration of visual information into text-based models can substantially improve model predictions, but so far only visual information extracted from static images has been used. In this paper, we consider the problem of grounding sentences describing actions in visual information extracted from videos. We present a general purpose corpus that aligns high quality videos with multiple natural language descriptions of the actions portrayed in the videos, together with an annotation of how similar the action descriptions are to each other. Experimental results demonstrate that a text-based model of similarity between actions improves substantially when combined with visual information from videos depicting the described actions.