EAGER: Scalable Video Retrieval

传统的视频分析研究一直集中在检测和识别任务的对象和活动，从已知的来源与相当狭窄的内容范围。这项工作将扩展可预测的双视图哈希算法在以前的工作从图像到视频。许多视频可以自然地与生产者和消费者评论（标签）的文本注释、使用语音到文本方法从语音轨迹导出的语言或与应用视觉模型（如人类检测器和本地活动检测器）相关联的语义词相关联。该团队将把基于联合收割机外观的视频分类方法与来自这些文本源的语言模型相结合，以便通过类似自然语言的界面检索视频。这将涉及研究如何将这些不同的文本源融合到一个向量空间语言模型中，然后将双视图哈希方法应用于视频数据库。然后，他们可以使用零镜头类别定义形式的文本代码来调查检索性能。这项研究是由情报分析师的需求驱动的，他们需要能够比传统的相关性反馈更有效地表达视频查询，并能够提供更具表达力的查询，包括名词和动词，就像人类语言一样。虽然仍然受到限制，但该方法在弥合传统的仅基于图像中假设关系的相关性反馈与全人类语言查询之间的差距方面还有很长的路要走。

Traditional video analysis research has been centered on detection and recognition tasks for objects and activities from known sources with a fairly narrow range of content. This effort would extend the predictable dual view hashing algorithm developed in previous work from images to videos. Many videos can be naturally associated with text annotations by the producer and consumer comments (tags), language derived from speech tracks using speech to text methods or the semantic words associated with applying vision models like human detectors and local activity detectors. The team will combine appearance based methods for video classification with language models derived from these text sources so that videos can be retrieved via a natural language like interface. This will involve investigating ways of fusing these different text sources in one vector space language model and then applying the dual view hashing methods to a database of videos. They can then investigate retrieval performance using the text codes for a form of zero shot category definition. The research is driven by the need for intelligence analysts to be able to express video queries more efficiently than traditional relevance feedback and to be able to provide more expressive queries that include nouns and verbs as they would with human language. While still constrained the approach goes a long way toward bridging the gap between traditional relevance feedback based only on assumed relationships in the image, and full human language queries.