SBIR Phase I: Multimodal Semantic Video Retrieval and Summarization

这个小型企业创新研究（SBIR）第一阶段项目旨在开发高度准确和高效的语义视频检索和摘要的方法和软件。视频语义将提供个性化的视频内容摘要，满足用户的喜好。这些摘要将基于镜头粒度，而不是广泛使用的基于关键帧的摘要，这些摘要忽略了语义。此外，所提出的技术将大大增强在线视频搜索，使用户能够检索语义相关的镜头，而不是整个视频。该软件的关键组件是一个自动语义视频注释系统，该系统集成了实时视频镜头检测、语音识别、自然语言处理和逻辑推理方法，可根据语义用户请求和偏好准确选择视频镜头。消费者和视频内容服务提供商将使用所提出的自适应视频消息传递技术，使用语义视频摘要消息（SVS）来有效地传达查询、偏好和结果。拟议中的软件一旦商业化，可能会影响在线视频内容搜索和检索方式的转变。此外，如果成功，该软件将推进构建视频摘要的最新技术，与当前技术相比，它可以准确地响应语义级别的用户查询。因此，软件可能是众多内容提供商和消费者感兴趣的，以在众多视频应用中采用。 该软件还可以集成到日益流行的数字录像机中，使所有者能够搜索大量的存档视频，并根据语义查询检索特定的视频，而不是通常不准确的文件名。另一方面，软件的独特摘要能力可以由内容/服务提供商使用，其中个性化的、基于语义的摘要标准可以由用户预定义，使得内容提供商，自适应（基于网络和设备功能）流摘要匹配用户的要求，以他们的智能手机或其他移动的设备。该奖项是根据2009年美国复苏和再投资法案资助（第111-5页）

This Small Business Innovation Research (SBIR) Phase I project aims to develop methodology and software for highly accurate and efficient semantic video retrieval and summarization. Video Semantics will, provide personalized summaries of video content that meet users' preferences. These summaries will be based on shot granularity instead of the widely used key-frame-based summaries that are oblivious of semantics. Additionally, the proposed technology will significantly enhance online video search by enabling users to retrieve only semantically-relevant shots instead of the entire video. The key component of the software is an automated semantic video annotation system that integrates realtime video shot detection, speech recognition, natural language processing, and logic inference methods to accurately select video shots according to semantic user requests and preferences. Consumers and video content service providers will use the proposed adaptive video messaging technique to efficiently communicate queries, preferences and results using Semantic Video Summary messages (SVS). The proposed software, once commercialized, can affect a shift in the way online video content is searched and retrieved. Moreover, if successful, the software will advance the state-of-the-art of constructing video summaries, which in contrast to current technologies, accurately responds to semantic level user queries. Consequently, the software may be of interest to numerous content providers and consumers to be employed in a multitude of video applications. The software could also be integrated into the ever-popular digital video recorders to enable the owner to search large volumes of archived videos and retrieve specific ones given semantic queries, rather than the usually inaccurate file names. On the other hand, the unique summarization capabilities of the software can be used by content/service providers where personalized, semantic-based summary criteria can be predefined by the user so that the content providers, adaptively (based on network and device capabilities) stream summaries matching users' requirements to their smart phones of other mobile devices.This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).