RI: Small: Temporal Causality For Video Event Analysis

该项目正在寻求一种新的策略，通过利用时间因果关系的统计测试来分析视频中的时间结构。动机是需要无监督的视频分析方法，这种方法不需要预定义的视频类别集或大量标记示例的语料库。起点是格兰杰因果关系的经典公式，它为两个时间序列之间的直接影响提供了原则性的统计检验。修改经典的成对格兰杰测试产生了一种适用于视频事件的方法，视频事件被表示为多点过程。使用这种表示，正在开发根据视觉单词随时间的交互作用将其分组的方法。这产生了一种新颖的自下而上的分割方法，可以在没有监督的情况下识别视觉单词之间的交互。进一步的目标是开发一种集成方法来建模视觉事件和识别因果关系。其他工作旨在开发由因果关系构建的新颖特征，以提高分类和检索任务的性能。总之，该项目正在开发新的无监督方法，用于基于时间因果分析来表示和分割视频。由此产生的算法提高了视频检索和分类任务的性能，并提供了组织和搜索 YouTube 视频等非结构化内容的新方法。用于视频分割和分类的新颖数据集以及分析软件库正在开发中，以促进研究界的采用。

This project is pursuing a novel strategy for the analysis of temporal structure in video through the exploitation of statistical tests of temporal causality. The motivation is the need for unsupervised video analysis methods which do not require a pre-defined set of video categories or a large corpus of labeled examples. The starting point is the classical formulation of Granger causality, which provides a principled statistical test for directed influence between two time series. Modifying the classical pair-wise Granger test leads to a method which is suitable for video events, which are represented as multiple point processes. Using this representation, methods are being developed for grouping visual words into sets based on their interaction over time. This results in a novel bottom-up segmentation approach which can identify interactions between visual words without supervision. A further goal is the development of an integrated approach to modeling visual events and identifying causal relations. Additional efforts are aimed at developing novel features constructed from causal relations with the goal of improved performance on categorization and retrieval tasks. In summary, the project is developing new unsupervised methods for representing and segmenting video based on temporal causal analysis. The resulting algorithms yield improved performance in video retrieval and categorization tasks, and provide new approaches to organizing and searching unstructured content such as YouTube videos. Novel datasets for video segmentation and categorization are being developed along with a library of analysis software to facilitate adoption by the research community.