SBIR Phase II: Filmmaking for Everyone: Computational Video Editing

这项小企业创新研究（SBIR）第二阶段项目的更广泛影响是改进用户生成的视频内容。在线视频平台从根本上改变了人们交流、学习和激励的方式。不幸的是，由于无法将它们编辑成引人注目的小插曲，许多潜在的鼓舞人心的视频丢失或从未共享。大多数编辑视频的工具都很昂贵，难以学习，而且耗时。该项目的研究使消费者或新兴企业能够通过使用计算电影摄影和深度学习和人工智能（AI）技术，只需点击一次手机，就可以制作精美的专业视频。交付的软件解决方案将在几分钟内生成高质量的编辑片段，而人工编辑则需要数小时。这个项目结合了计算机视觉对视频的分析和编辑算法，使新的创作者能够参与到这个快速发展的媒体中来。这个小企业创新研究（SBIR）二期项目将生产人工智能驱动的软件，用于在手机平台上自动将原始视频片段编辑成高质量的短片。提议的项目将集成先进的计算视频处理、计算机视觉和音频识别。原型AI编辑器将从源素材中选择相关内容，并将其与音乐同步，仅使用典型移动平台有限的计算资源。人工智能根据视频内容、音乐内容和从大量类似电影数据集中学习的叙事编辑风格做出决策。该项目将提供基于人工智能的编辑技术，根据输入视频中的口语对话对输入镜头进行修剪和安排。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impacts of this Small Business Innovation Research (SBIR) Phase II project enable improved user-generated video content. Video online platforms have radically changed how people communicate, learn, and inspire. Unfortunately, many potentially inspiring videos are lost or never shared due to an inability to edit them into compelling vignettes. Most tools for editing video are expensive, difficult to learn, and time-consuming. This project’s research enables consumers or nascent businesses to make polished, professional videos with a single phone click through the use of computational cinematography and techniques from deep learning and artificial intelligence (AI). The delivered software solution will produce high-quality edited footage within minutes, compared with a human editor requiring hours. This project combines the analysis of video using computer vision with editing algorithms to empower new creators to participate in this fast-growing medium. This Small Business Innovation Research (SBIR) Phase II project will produce AI-powered software for automatically editing raw video footage into quality short films on a mobile phone platform. The proposed project will integrate advanced computational video manipulation, computer vision, and audio recognition. The prototype AI editor will select relevant content from source footage and synchronize it to music, using only the restricted computational resources of a typical mobile platform. The AI makes decisions based on the video content, the music content, and narrative editing styles learned from a large dataset of similar films. This project will deliver AI-based editing technology that trims and arranges input footage based on the spoken dialogue in the input videos.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.