Spatiotemporal Models for Analyzing and Understanding Video

视频包含了丰富的信息来理解和操纵周围的世界;然而，在其原始形式作为一组逐点强度，视频无法使明确的相关信息可用于进一步处理。 拟议的研究计划旨在探索从原始视频流中提取有用信息的方法，以便从处理的最早阶段开始进行进一步分析。 在今后五年中，将审查以下三个基本调查问题。 是什么使得人类动作的有用部分分解用于在视频流中自动识别它们？ 如何同时利用视频中的外观和动态信息来恢复成像场景的三维几何布局？ 观察到的一群密集的个体的集体行为如何帮助跟踪每个个体？ 这三个基本问题将通过时间图像的数学模型的紧密耦合，新算法的开发（例如，图像处理）和利用受控的合成和真实的数据的系统经验评估。 这项研究在三个主要方面具有创新性。 首先，该研究计划为我们对原始视频中可用信息的基本理解提供了广泛的理论见解。 其次，这些见解为解决各种实际的基于视觉的应用的算法的开发提供了坚实的基础。 例如，动作识别可用于视频索引和浏览，3D结构恢复是开发自主机器人的关键部分，在人口密集的大型区域进行跟踪可以帮助安全，安全和公共空间设计。 第三，它为学生提供了在计算机视觉领域以及更广泛的新兴数据科学领域发展技术和分析技能的机会。 这项培训为学生在学术或工业研究和开发的职业生涯做好准备。

Video contains a wealth of information for understanding and manipulating the surrounding world; however, in its raw form as a set of pointwise intensities, video fails to make explicit pertinent information available for further processing. The proposed research program aims to explore methods to extract useful information from raw video streams for further analysis, starting at the earliest stages of processing. Over the next five years, the following three basic questions of inquiry will be examined. What makes a useful part decomposition of human actions for automatically recognizing them in a video stream? How can appearance and dynamic information in video be used simultaneously to recover the three dimensional geometric layout of an imaged scene? How can the observed collective behaviours of a densely packed ensemble of individuals aid in tracking each individual? These three basic questions will be addressed via a tight coupling of mathematical models of temporal imagery, the development of novel algorithms (e.g., image processing) and systematic empirical evaluation with both controlled synthetic and real data. The proposed research is innovative in three primary ways. First, the research program offers broad theoretical insights into our fundamental understanding of information available in raw video. Second, these insights provide the firm foundation for the development of algorithms addressing a variety of practical vision-based applications. For example, action recognition can be used for video indexing and browsing, 3D structure recovery is a key part of developing autonomous robots, and tracking in large densely populated areas can aid in security, safety and public space design. Third, it gives students the opportunity to develop their technical and analytical skills in the area of computer vision and more generally the emerging field of data science. This training prepares students for careers in academic or industrial research and development.