I-Corps: Semantic Video - from Video to Descriptions

这个I-Corps项目的更广泛的影响/商业潜力涉及视频的计算机视觉分析，使用视觉和听觉线索来创建内容的描述。该技术具有广泛的潜在应用，从执法到监控再到消费者应用。其中包括能够有效存储和检索大量相机数据。 智能监控系统可以通过允许将一整天的视频片段汇总为安全相关事件列表的功能进行增强。 I-Corps项目的内容是，对以自然语言文本和语法（语义）表达视频内容的计算机视觉技术进行商业化可行性研究。该项目建立在一个视频分析框架上，该框架利用最先进的方法进行对象检测和动作识别，并以统一的形式主义编码，该形式主义以数学和统计方法（称为模式理论）进行编码。视频分析方法可以（i）处理复杂事件的结构可变性，而不需要大的训练数据，同时利用容易获得的本体信息，（ii）克服动作和对象的机器学习分类器的分类错误，（iii）适应场景混乱，即不存在于场景中的活动中的无关对象，（iv）并且管理基本事件的序列，都没有再培训。的形式主义允许容易纳入时间，空间和逻辑约束。该团队已经在标准数据集上展示了该系统，用于对人类活动识别任务的计算机视觉性能进行基准测试。

The broader impact/commercial potential of this I-Corps project involves computer vision analysis of video, using both visual and auditory cues, to create descriptions of the content. The technology has a large variety of potential applications from law enforcement to surveillance to consumer applications. These include enabling the efficient storage and retrieval of large volumes of camera data. Smart surveillance systems can be enhanced with features that allows for summarization of daylong video footages as a list of security-relevant events. The technology can also allow automated organization of large collections of multimedia data.This I-Corps project involves commercialization feasibility research for a computer vision technology for expressing video content in terms of natural language text and grammar, i.e. semantics. This project builds on a video analysis framework that leverages state-of-the-art methods for object detection and action recognition in a unified formalism encoded in terms of a mathematical and statistical approach known as pattern theory. The video analysis approach can (i) handle structural variability of complex events without requiring large training data while exploiting easily available ontological information, (ii) overcome classification errors of machine learning classifiers of actions and objects, (iii) accommodate scene clutter, i.e. extraneous objects that do not in the activity present in the scene, (iv) and manage sequences of elementary events, all without retraining. The formalism allows for the easy incorporation of temporal, spatial, and logical constraints. This team has demonstrated this system on standard datasets used to benchmark performance in computer vision for human activity recognition tasks.