Spatiotemporal Models for Analyzing and Understanding Video

点击翻译按钮获取中文摘要

Video contains a wealth of information for understanding and manipulating the surrounding world; however, in its raw form as a set of pointwise intensities, video fails to make explicit pertinent information available for further processing. The proposed research program aims to explore methods to extract useful information from raw video streams for further analysis, starting at the earliest stages of processing. Over the next five years, the following three basic questions of inquiry will be examined. What makes a useful part decomposition of human actions for automatically recognizing them in a video stream? How can appearance and dynamic information in video be used simultaneously to recover the three dimensional geometric layout of an imaged scene? How can the observed collective behaviours of a densely packed ensemble of individuals aid in tracking each individual? These three basic questions will be addressed via a tight coupling of mathematical models of temporal imagery, the development of novel algorithms (e.g., image processing) and systematic empirical evaluation with both controlled synthetic and real data. The proposed research is innovative in three primary ways. First, the research program offers broad theoretical insights into our fundamental understanding of information available in raw video. Second, these insights provide the firm foundation for the development of algorithms addressing a variety of practical vision-based applications. For example, action recognition can be used for video indexing and browsing, 3D structure recovery is a key part of developing autonomous robots, and tracking in large densely populated areas can aid in security, safety and public space design. Third, it gives students the opportunity to develop their technical and analytical skills in the area of computer vision and more generally the emerging field of data science. This training prepares students for careers in academic or industrial research and development.