RI: Small: Modeling and Parsing Time Series for Causal Analysis with Application to Action Interpretation in Video of Natural and Man-made Environments

点击翻译按钮获取中文摘要

This project tackles the development of new tools for the semantic analysis of temporal signals, in particular (but not restricted to) video sequences. While most of the emphasis in video analysis so far has been at the low-level, the investigators plan to explore the use of Causal Analysis to perform inference and decisions to analyze video signals. The challenge in this project is to bridge the gap between basic descriptor at the signal level and Causal Calculus, that acts on semantically meaningful representations. In particular, long-range prediction, not just short-range continuous extrapolation, requires the development of new tools that allow "interventions" into the model. How would the state "X" evolve if event "Y" were to occur? To attain the goals set forth in the proposal, the investigators must tackle fundamental problems in the analysis of time series, both at the low/mid-level (defining a proper notion of ``distance'' between time series that respects their intrinsic dynamics), at the mid-level (defining clustering schemes for action segments), and at the high-level (develop action semantics in an abductive framework). During this pilot one-year project, the investigators plan to explore the feasibility of using causal analysis for performing long-range temporal prediction of events and actions from visual data. Sample applications that are impacted in case of success are broad ranging from surveillance to environmental monitoring to driver assistance in transportation, with significant societal impact in reducing traffic accidents.