RI: Small: Modeling and Parsing Time Series for Causal Analysis with Application to Action Interpretation in Video of Natural and Man-made Environments

该项目致力于开发用于时间信号语义分析的新工具，特别是（但不限于）视频序列。虽然迄今为止视频分析的大部分重点都处于低水平，但研究人员计划探索使用因果分析来执行推理和决策来分析视频信号。该项目的挑战是弥合信号级别的基本描述符和因果演算之间的差距，因果演算作用于语义上有意义的表示。特别是，长期预测，而不仅仅是短期连续外推，需要开发允许对模型进行“干预”的新工具。如果事件“Y”发生，状态“X”将如何演变？为了实现提案中提出的目标，研究人员必须解决时间序列分析中的基本问题，包括低/中层（定义尊重其内在动态的时间序列之间“距离”的适当概念）、中层（定义动作片段的聚类方案）和高层（在溯因框架中开发动作语义）。在这个为期一年的试点项目中，研究人员计划探索使用因果分析根据视觉数据对事件和动作进行长期时间预测的可行性。如果成功，受影响的示例应用范围广泛，从监视到环境监测再到运输中的驾驶员辅助，在减少交通事故方面具有重大的社会影响。

This project tackles the development of new tools for the semantic analysis of temporal signals, in particular (but not restricted to) video sequences. While most of the emphasis in video analysis so far has been at the low-level, the investigators plan to explore the use of Causal Analysis to perform inference and decisions to analyze video signals. The challenge in this project is to bridge the gap between basic descriptor at the signal level and Causal Calculus, that acts on semantically meaningful representations. In particular, long-range prediction, not just short-range continuous extrapolation, requires the development of new tools that allow "interventions" into the model. How would the state "X" evolve if event "Y" were to occur? To attain the goals set forth in the proposal, the investigators must tackle fundamental problems in the analysis of time series, both at the low/mid-level (defining a proper notion of ``distance'' between time series that respects their intrinsic dynamics), at the mid-level (defining clustering schemes for action segments), and at the high-level (develop action semantics in an abductive framework). During this pilot one-year project, the investigators plan to explore the feasibility of using causal analysis for performing long-range temporal prediction of events and actions from visual data. Sample applications that are impacted in case of success are broad ranging from surveillance to environmental monitoring to driver assistance in transportation, with significant societal impact in reducing traffic accidents.