Automatic consumer video summarization by audio and visual analysis
Automatic consumer video summarization by audio and visual analysis
复制标题
DOI：
10.1109/icme.2011.6011841
复制
发表时间：
2011-07
期刊：
影响因子：
--
通讯作者：
Wei Jiang;Courtenay V. Cotton;A. Loui
中科院分区：
文献类型：
--
作者：
Wei Jiang;Courtenay V. Cotton;A. Loui
视频摘要通过分析视频内容来提供视频流的压缩版本。消费者视频的自动摘要是一个重要的工具，它有助于在大型消费者视频集合中高效地浏览、搜索和创建相册。本文研究了消费者领域的自动视频摘要，由于内容分析的挑战性问题，即，消费者视频是在不受控制的条件下捕获的，诸如不均匀照明、杂乱和大的摄像机运动，并且在严重噪声下具有作为多个声源的混合的低质量声轨。为了追求可靠的总结，与实际的消费者用户的案例研究进行，从中获得一套面向消费者的指导方针。这些准则反映了视觉和音频方面的高级语义规则，消费者认为这些规则对制作好的视频摘要很重要。根据这些准则，开发了一种自动视频摘要算法，其中使用视觉和音频信息来生成改进的摘要。据我们所知，这是第一次系统地研究消费者质量的视频自动摘要。消费者主体的实验评估表明了我们的方法的有效性。
Video summarization provides a condensed version of a video stream by analyzing the video content. Automatic summarization of consumer videos is an important tool that facilitates efficient browsing, searching, and album creation in large consumer video collections. This paper studies automatic video summarization in the consumer domain where most previous methods cannot be easily applied due to the challenging issues for content analysis, i.e., consumer videos are captured with uncontrolled conditions such as uneven illumination, clutter, and large camera motion, and with poor-quality soundtrack as a mix of multiple sound sources under severe noise. To pursue reliable summarization, a case study with actual consumer users is conducted, from which a set of consumer-oriented guidelines is obtained. The guidelines reflect the high-level semantic rules, in both visual and audio aspects, which are recognized by consumers as important to produce good video summaries. Following these guidelines, an automatic video summarization algorithm is developed where both visual and audio information are used to generate improved summaries. To the best of our knowledge, this is a first systematic study on automatic summarization of consumer-quality videos. Experimental evaluations from consumer subjects show the effectiveness of our approach.