RI: Medium: Collaborative Research: Learning to Summarize User-Generated Video

RI：媒介：协作研究：学习总结用户生成的视频

• 批准号: 1513966
• 负责人: Fei Sha
• 金额: $ 53.42万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2016-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1513966&HistoricalAwards=false
• 关键词: RI; Medium; Collaborative; Research; Learning

Today there is far more video being captured - by consumers, scientists, defense analysts, and others - than can ever be watched. With this explosion of video data comes a pressing need to develop automatic video summarization algorithms. Video summarization takes a long video as input and produces a short video as output, while preserving its information content as much as possible. As such, summarization techniques have great potential to make large video collections substantially more efficient to browse, search, disseminate, and facilitate communication. Such increased efficiency will play a vital role in many important application areas. For example, with reliable summarization systems, a primatologist gathering long videos of her animal subjects could quickly browse a week's worth of their activity before deciding where to inspect the data most closely. A young student searching YouTube to learn about Yellowstone National Park could see at a glance what content exists, much better than today's simple thumbnail images can depict. An intelligence agent could rapidly sift through reams of aerial video, reducing the resources required to analyze surveillance data to identify suspicious activities.This project develops new machine learning and computer vision algorithms for video summarization. Unsupervised methods, which are the cornerstone of nearly all existing approaches, have become increasingly limiting due to their reliance on hand-crafted heuristics. By instead posing video summarization as a supervised learning problem, this project investigates a markedly different formulation of the task. The research team is investigating four key new ideas: powerful probabilistic models for learning to select the optimal subset of video frames for summarization, semi-supervised learning models and co-summarization algorithms for leveraging the abundance of multiple related videos, algorithms for exploiting photos on the Web to improve summarization, and evaluation protocols that assess summaries in a way that aligns well with human comprehension. The broader impact of the proposed research includes practical tools for video summarization, scientific advances that appeal broadly to several communities, publicly disseminated research results, inter-disciplinarily trained graduate students, and outreach activities to engage young students in STEM education and career paths.

如今，消费者、科学家、国防分析人士和其他人捕捉到的视频远比以往任何时候都要多。随着视频数据的爆炸式增长，迫切需要开发自动视频摘要算法。视频摘要以一段长视频为输入，输出一段短视频，同时尽可能地保留其信息内容。因此，摘要技术具有很大的潜力，可以使大型视频集合更有效地浏览、搜索、传播和促进交流。这种效率的提高将在许多重要的应用领域发挥至关重要的作用。例如，有了可靠的总结系统，灵长类动物学家收集她的动物实验对象的长视频，可以在决定最仔细检查数据的地方之前快速浏览一周的活动。一个年轻的学生在YouTube上搜索黄石国家公园，可以一眼看到存在的内容，这比今天简单的缩略图所能描述的要好得多。情报人员可以快速筛选大量空中视频，减少分析监控数据以识别可疑活动所需的资源。本项目为视频摘要开发新的机器学习和计算机视觉算法。无监督方法是几乎所有现有方法的基石，由于它们依赖于手工制作的启发式，已经变得越来越有限。通过将视频摘要作为一个监督学习问题，本项目研究了一个明显不同的任务公式。研究小组正在研究四个关键的新思想：用于选择视频帧的最佳子集进行总结的强大概率模型，用于利用大量相关视频的半监督学习模型和共同总结算法，用于利用网络上的照片来改进摘要的算法，以及以与人类理解相一致的方式评估摘要的评估协议。拟议研究的更广泛影响包括用于视频总结的实用工具，广泛吸引多个社区的科学进步，公开传播的研究成果，跨学科培训的研究生，以及吸引年轻学生参与STEM教育和职业道路的外展活动。