RI: Medium: Collaborative Research: Learning to Summarize User-Generated Video

RI：媒介：协作研究：学习总结用户生成的视频

• 批准号: 1514118
• 负责人: Kristen Grauman
• 金额: $ 54.7万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1514118&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Learning

Today there is far more video being captured - by consumers, scientists, defense analysts, and others - than can ever be watched. With this explosion of video data comes a pressing need to develop automatic video summarization algorithms. Video summarization takes a long video as input and produces a short video as output, while preserving its information content as much as possible. As such, summarization techniques have great potential to make large video collections substantially more efficient to browse, search, disseminate, and facilitate communication. Such increased efficiency will play a vital role in many important application areas. For example, with reliable summarization systems, a primatologist gathering long videos of her animal subjects could quickly browse a week's worth of their activity before deciding where to inspect the data most closely. A young student searching YouTube to learn about Yellowstone National Park could see at a glance what content exists, much better than today's simple thumbnail images can depict. An intelligence agent could rapidly sift through reams of aerial video, reducing the resources required to analyze surveillance data to identify suspicious activities.This project develops new machine learning and computer vision algorithms for video summarization. Unsupervised methods, which are the cornerstone of nearly all existing approaches, have become increasingly limiting due to their reliance on hand-crafted heuristics. By instead posing video summarization as a supervised learning problem, this project investigates a markedly different formulation of the task. The research team is investigating four key new ideas: powerful probabilistic models for learning to select the optimal subset of video frames for summarization, semi-supervised learning models and co-summarization algorithms for leveraging the abundance of multiple related videos, algorithms for exploiting photos on the Web to improve summarization, and evaluation protocols that assess summaries in a way that aligns well with human comprehension. The broader impact of the proposed research includes practical tools for video summarization, scientific advances that appeal broadly to several communities, publicly disseminated research results, inter-disciplinarily trained graduate students, and outreach activities to engage young students in STEM education and career paths.

如今，消费者、科学家、国防分析师和其他人拍摄的视频比以往任何时候都多。 随着视频数据的爆炸式增长，迫切需要开发自动视频摘要算法。 视频摘要将长视频作为输入，并产生短视频作为输出，同时尽可能地保留其信息内容。 因此，摘要技术具有很大的潜力，使大型视频集合更有效地浏览，搜索，传播，并促进通信。 这种提高的效率将在许多重要的应用领域发挥至关重要的作用。 例如，有了可靠的摘要系统，一位灵长类动物学家在收集动物实验对象的长视频时，可以快速浏览动物一周的活动，然后再决定在哪里最接近地检查数据。 一个年轻的学生搜索YouTube来了解黄石国家公园，可以一眼看到存在的内容，比今天简单的缩略图更好地描述。 智能代理可以快速筛选大量的空中视频，减少分析监控数据以识别可疑活动所需的资源。该项目开发了用于视频摘要的新机器学习和计算机视觉算法。 无监督方法是几乎所有现有方法的基石，由于依赖于手工制作的算法，因此变得越来越有限。 通过将视频摘要作为一个监督学习问题，该项目研究了一个明显不同的任务制定。 该研究小组正在研究四个关键的新想法：用于学习选择最佳视频帧子集进行摘要的强大概率模型，用于利用多个相关视频的丰富性的半监督学习模型和共同摘要算法，用于利用网络上的照片来改进摘要的算法，以及以符合人类理解的方式评估摘要的评估协议。 拟议研究的更广泛影响包括视频摘要的实用工具，广泛吸引多个社区的科学进步，公开传播的研究成果，跨学科培训的研究生，以及让年轻学生参与STEM教育和职业道路的外展活动。