Model-Based Segmentation, Structuring, and Display of Extended Video Sequences

这项研究创建和评估算法，表示和显示结构，分析-并帮助人类视频浏览器理解和搜索-扩展（半小时或更长时间）视频序列的场景和主题结构。 这些算法和表示法将戏剧、喜剧、新闻广播、体育赛事和脱口秀节目分成三个具有结构意义的层次：镜头、场景或场景集，以及场景或行为的主题重复，从而提供了一种手段来重新捕捉视频轮廓的原始“故事板”层次。 这三个层次的语义连贯的单位，然后显示在一个新的乐谱样的图形格式，用户变量的精度和细节。 这项工作探讨了四个方面的分析：选择适当的低层次的图像功能，设计的表示和结构的关系，在每一个可调的人类记忆模型的基础上，在三个层次，探索显示配置和视频子序列选择有效的用户浏览，并验证算法优化和用户反馈的大型语料库的视频。 这项工作明确了一些结构规则和技巧，管理主题连贯的长视频序列的创建。 在教育上，这些问题已经激发了一个新的高年级课程，“计算机视觉界面”。

This research creates and evaluates algorithms, representations, and display structures that analyze---and help the human video browser understand and search---the scenic and thematic structure of extended (half-hour or more) video sequences. These algorithms and representations segment dramas, comedies, newscasts, sporting events, and talk shows into three levels of structural significance: camera shots; scenes, or sets; and the thematic repetitions of scenes, or acts, thus providing a means to recapture something like the originating ``storyboard'' level of the video's outline. These three levels of semantically coherent units are then displayed in a novel musical score-like graphic format, with user variable precision and detail. This work explores four aspects of analysis: the selection of appropriate low-level image features, the design of representations and relationships of structure at each of the three levels based on tunable human memory models, the exploration of display configurations and video subsequence selection for effective user browsing, and the validation of algorithmic optimizations and user feedback over a large corpus of videos. This work makes explicit some of the structural rules and heuristics that govern the creation of thematically coherent long video sequences. Educationally, these issues have already inspired a new upper division course, "Visual Interfaces to Computers".