Deep Video Analysis

在过去十年中，由于图像捕获设备(例如移动电话)、高容量数据存储设备和与网络相关的视频服务的广泛使用，我们经历了捕获视频的指数级增长。视频包含了丰富的信息，用于理解和操纵周围的世界；然而，在其原始形式下，它是没有意义的。计算机视觉提供了一种自动理解和组织视频流中潜在信息的过程的方法。*我提出的研究计划的目标是促进我们对视频与周围世界相关的基本理解(例如，“周围世界的什么几何信息可以从视频中恢复？”)，通过利用视频的冗余和可预测的性质来开发用于视频推理的高效体系结构(“我可以重复使用哪些先前的计算来代替新的计算？”)，并开发用于预测未来的原始RGB帧或给定过去的视频帧的语义像素级标签的模型(给出我到目前为止所看到的，接下来是什么图像？“)。我的研究计划背后的一个关键主题是在开发深层多层可训练体系结构时利用领域知识和约束。*我提出的计划在以下方面对计算机视觉研究领域和加拿大有利。首先，我的程序探索了在原始视频中提供明确相关信息的方法。在其核心，我们试图揭示支持视频理解的基本原则，以指导自动化处理方法的发展。其次，这项研究可以支持大量现实世界中有影响力的应用，包括自动驾驶、视频挖掘和组织以及安全监控。最后，我的研究计划在快速发展的计算机视觉、机器学习以及更广泛的数据科学领域提供了有价值的、高质量的培训。在加拿大的工业研发和学术环境中，人们越来越意识到对这些技能的需求；这使得拥有这些技能的个人非常受欢迎。

Over the past decade, we have experienced an exponential growth in captured videos due to the broad availability of image capture devices (e.g., mobile phones), high capacity data storage devices, and web-related video services. Video contains a wealth of information for understanding and manipulating the surrounding world; however, in its raw form it is meaningless. Computer vision provides a means for automating the process of understanding and organizing latent information in video streams.******The goals of my proposed research program are to advance our fundamental understanding of video as it relates to the surrounding world (e.g., "What geometric information of the surrounding world is recoverable from video?"), develop efficient architectures for video inference by leveraging the redundant and predictable nature of video ("What previous computations can I reuse in place of new computations?"), and develop models for predicting future raw RGB frames or semantic pixel-wise labels given past video frames ("Given what I have seen so far, what imagery comes next?"). A key theme underlying my research program is leveraging domain knowledge and constraints in the development of deep multi-layer trainable architectures.******My proposed program benefits the field of computer vision research and Canada in the following ways. First, my program explores methods for making explicit pertinent information available within raw video. At its core, we are trying to uncover basic principles underpinning video understanding to guide the development of automated processing methods. Second, there are a multitude of real-world, impactful applications that can be supported by this research, including autonomous driving, video mining and organization, and security surveillance. Finally, my research program provides valuable, high quality training in the rapidly evolving areas of computer vision, machine learning, and more broadly data science. There is a growing awareness of the need for these skill-sets in both Canadian industrial research and development, and academic settings; making individuals with these skill-sets highly sought after.**