III-COR: Analysis and Display of Semantics of Structured and Unstructured Videos

该项目探索了三种新的相关方法，以使视频的索引和检索（1）通过开发快速特征选择更有效，（2）通过设计可测量有用的索引本体更有意义，（3）通过展示集成的多媒体浏览器更人性化，第一种方法包括机器学习算法对过度冗余数据的启发式适应，视频. 第二种方法包括使用统计工具来检查可以附加到视频片段的描述性标签，以测量和改进其质量。 第三种方法包括利用前两种方法来发现和显示即使在未经编辑的学生演示视频中也隐藏的弱结构。 该实验研究将通过持续的用户研究来完善，涉及初中，高中，大学和研究生的学生。该项目的结果将在计算机视觉，机器学习，数据管理，信息检索，本体设计，用户界面技术和用户研究的多个交叉点提供进步，在更广泛的感官检索领域中具有可能的应用。更广泛的影响：我们期望该浏览器能够准确、快速地查看教师和学生录制的演示文稿，从而提高本科教育的有效性。潜在的新技术将允许实时分析和访问更多的标准视频。 该项目的网址（http：//www.cs.columbia.edu/publicjrk/unstructured）将用于传播由此产生的出版物、开放源码以及关于如何获得附加说明的录像数据集的说明。

This project explores three new related approaches to making the indexing and retrieval of videos (1) more efficient by developing rapid feature selection, (2) more meaningful by devising measurably useful indexing ontologies, and (3) more humanly navigable by demonstrating integrated multimedia browsers, even when the videos are unedited.The first approach consists of heuristic adaptations of machine learning algorithms to the overly redundant data that is video. The second approach consists of using statistical tools to examine the descriptive tags that can be affixed to video segments, in order to measure and refine their quality. The third approach consists of exploiting the first two approaches to discover and display the weak structure that is latent even in the unedited videos of student presentations. The experimental research will be refined by continuing user studies, involving students from middle school, high school, college, and post-graduates.The results of this project will provide advances at the multiple intersections of computer vision, machine learning, data management, information retrieval, ontology design, user interface technology, and user studies, with possible applications in the wider areas of sensory retrieval in general.Broader impacts: We expect that the browser will enhance the effectiveness of undergraduate education by allowing accurate and rapid review of both instructor and student recorded presentations.We also expect that the underlying novel technologies will permit real-time analysis and access of more standard videos. The project Web site (http://www.cs.columbia.edu/~jrk/unstructured) will be used to disseminate resulting publications, open-source code, and instructions on how to obtain annotated video data sets.