Deep Video Analysis

在过去的十年中，由于图像捕获设备（如移动电话）、大容量数据存储设备和与网络相关的视频服务的广泛可用性，我们经历了视频捕获的指数级增长。视频包含了理解和操纵周围世界的丰富信息；然而，它的原始形式是没有意义的。计算机视觉为自动理解和组织视频流中的潜在信息提供了一种手段。******我提出的研究计划的目标是推进我们对视频的基本理解，因为它与周围世界有关（例如，“周围世界的哪些几何信息可以从视频中恢复？”），通过利用视频的冗余性和可预测性，为视频推理开发高效的体系结构（“我可以重用哪些先前的计算来代替新的计算？”），并开发模型来预测未来的原始RGB帧或基于过去视频帧的语义像素标签（“根据我到目前为止看到的，接下来会有什么图像？”）。我的研究计划的一个关键主题是在开发深层多层可训练架构时利用领域知识和约束。******我提出的计划在以下方面对计算机视觉研究领域和加拿大有益。首先，我的计划探讨了在原始视频中提供明确相关信息的方法。其核心是，我们试图揭示支撑视频理解的基本原理，以指导自动化处理方法的发展。其次，这项研究可以支持许多现实世界中有影响力的应用，包括自动驾驶、视频挖掘和组织以及安全监控。最后，我的研究项目在快速发展的计算机视觉、机器学习和更广泛的数据科学领域提供了有价值的、高质量的培训。越来越多的人意识到加拿大工业研发和学术环境对这些技能的需求；使拥有这些技能的人备受追捧

Over the past decade, we have experienced an exponential growth in captured videos due to the broad availability of image capture devices (e.g., mobile phones), high capacity data storage devices, and web-related video services. Video contains a wealth of information for understanding and manipulating the surrounding world; however, in its raw form it is meaningless. Computer vision provides a means for automating the process of understanding and organizing latent information in video streams.******The goals of my proposed research program are to advance our fundamental understanding of video as it relates to the surrounding world (e.g., "What geometric information of the surrounding world is recoverable from video?"), develop efficient architectures for video inference by leveraging the redundant and predictable nature of video ("What previous computations can I reuse in place of new computations?"), and develop models for predicting future raw RGB frames or semantic pixel-wise labels given past video frames ("Given what I have seen so far, what imagery comes next?"). A key theme underlying my research program is leveraging domain knowledge and constraints in the development of deep multi-layer trainable architectures.******My proposed program benefits the field of computer vision research and Canada in the following ways. First, my program explores methods for making explicit pertinent information available within raw video. At its core, we are trying to uncover basic principles underpinning video understanding to guide the development of automated processing methods. Second, there are a multitude of real-world, impactful applications that can be supported by this research, including autonomous driving, video mining and organization, and security surveillance. Finally, my research program provides valuable, high quality training in the rapidly evolving areas of computer vision, machine learning, and more broadly data science. There is a growing awareness of the need for these skill-sets in both Canadian industrial research and development, and academic settings; making individuals with these skill-sets highly sought after.**