SBIR Phase II: Multimodal Semantic Video Retrieval and Summarization

SBIR 第二阶段：多模态语义视频检索和摘要

• 批准号: 1058428
• 负责人: Wael Abd-Almageed
• 金额: $ 50万
• 依托单位: Video Semantics LLC
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-04-01 至 2013-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1058428&HistoricalAwards=false
• 关键词: SBIR; Phase; II; Multimodal; Semantic

This Small Business Innovation Research Phase II project will develop contextual video segmentation and automatic tagging technology and software. In long video streams that contain one or more topics, the software automatically discovers the beginnings and ends of Contextually-Coherent Video Segments in each video. Moreover, Video Semantics' technology automatically assigns textual tags to each segment such that these tags describe the topic discussed in that segment. The tags assigned make all parts of the video easily searchable. Large video producers currently depend on manually segmenting their content into small segments and assigning textual tags to these segments in order to make them searchable. A short advertisement is then inserted before each segment. This manual segmentation and tagging process represents a significant pain point for content producers because it is labor intensive and not cost effective. Meanwhile, government agencies, which continuously monitor video content depend on speech recognition to spot specified keywords. This approach inflicts two pain points: (i) analysts have to deal with large number of false detections because the context in which the keyword occurs might be irrelevant, and (ii) if the keyword occurs in an important context, analysts still need to scroll back and forth into the video to find the beginning of the relevant segment. Video Semantics' technology and products have the potential to efficiently address significant market needs. In addition to the commercial applications, the proposed technology will enable media monitoring agencies to perform their tasks more efficiently saving valuable analyst time and resources. Moreover, because Video Semantics? technology is language-independent, media monitoring agencies will be able to monitor more content in foreign languages without the need to develop language-specific technologies. The company will employ an indirect sales strategy via partnerships with software companies that develop media monitoring solutions and metadata generation tools. The company has identified its first customer and is working with them to integrate the contextual segmentation and tagging technology with their current media monitoring solutions.

这个小型企业创新研究第二阶段项目将开发上下文视频分割和自动标记技术和软件。在包含一个或多个主题的长视频流中，软件自动发现每个视频中上下文相关视频片段的开始和结束。此外，Video Semantics的技术自动为每个片段分配文本标签，以便这些标签描述该片段中讨论的主题。分配的标签使视频的所有部分都易于搜索。 大型视频制作商目前依赖于手动将其内容分割成小片段，并为这些片段分配文本标签，以便使其可搜索。然后在每个片段之前插入一个简短的广告。这种手动分割和标记过程对内容制作者来说是一个很大的痛点，因为它是劳动密集型的，而且不符合成本效益。与此同时，持续监控视频内容的政府机构依靠语音识别来识别特定的关键词。这种方法造成了两个痛点：（i）分析师必须处理大量的错误检测，因为关键字出现的上下文可能是不相关的，以及（ii）如果关键字出现在重要的上下文中，分析师仍然需要来回滚动到视频中以找到相关片段的开始。 Video Semantics的技术和产品有潜力有效地满足重要的市场需求。除了商业应用外，拟议的技术还将使媒体监测机构能够更有效地执行任务，节省宝贵的分析时间和资源。此外，因为视频语义？由于这种技术是独立于语言的，媒体监测机构将能够监测更多的外语内容，而不需要开发特定语言的技术。该公司将通过与开发媒体监控解决方案和元数据生成工具的软件公司合作，采用间接销售策略。该公司已经确定了第一个客户，并正在与他们合作，将上下文分割和标记技术与他们目前的媒体监控解决方案相集成。