Hierarchical Representation and Analysis of Visual Spacetime

基于视觉的系统是计算、机器人学和人工智能领域的中心研究课题。随着汽车、消费和家庭技术(例如Amazon Echo、Google Home、Apple HomePod)的当前趋势正在利用视频数据来了解环境并对其做出反应，这一突出地位应运而生。此外，当我们试图理解如何从仅仅捕获的视频中恢复关于世界的详细信息时，有趣的科学问题出现了。我的研究目标是支持视频理解的理论和工程原理。*我对视觉信息的具体研究为深入了解图像(例如视频)的时间序列的基本属性提供了见解，并通过指示新的应用来扩展我们的技术能力。我使用数学技术来分析图像，根据分析结果开发处理算法，并进行实证测试以评估其性能。这种方法允许我们提出和回答有关视频和其他时间图像序列来源的基本问题。感兴趣的问题包括：如何根据可视内容组织、归档和搜索复杂的视频流？如何使用捕获的场景的多个视图来重建场景的三维模型？关键的新颖性来自于系统规范中不同寻常的显式分析细节，与大多数其他系统相比，这提供了系统如何操作的清晰度的好处，并大大减少了对训练数据的需求。*我的研究在三个重要方面对加拿大做出了贡献。首先，它推动了计算机视觉学科的发展，因为我们揭示了图像信息的基本属性，并开发了相应的算法，用于整合到视觉机器中；这一进步使加拿大能够在关键的科学和技术领域取得成功。其次，它为学生提供了重要调查领域的实践经验和技能：计算机视觉、视频处理，更广泛地说，计算机科学和工程。学生们继续在科学、技术、教学和其他领域从事职业，同时也带来了参与研究的丰富经验。第三，它为在实践中开发图像数据的新方法铺平了道路。令人感兴趣的应用包括用于从互联网到个人手持设备的日常事务的视频处理，使内容更容易理解的视频增强，以及用于机器人和自动驾驶车辆的智能处理模块。因此，这项工作使这些技术和服务的生产者和消费者都受益。

Vision-based systems are a central research topic in computing, robotics and artificial intelligence. This prominence arises as current trends in automotive, consumer and home technology (e.g., Amazon Echo, Google Home, Apple HomePod) are leveraging video data to understand and react to the environment. Moreover, interesting scientific questions arise as we seek to understand how it is possible to recover detailed information about the world from a mere captured video. My research targets the underpinning theoretical and engineering principles of video understanding.******My particular studies of visual information provide insights into the fundamental properties of temporal sequences of images (e.g., video) and extend our technological capabilities by indicating novel applications. I use mathematical techniques to analyze images, develop processing algorithms based on the resulting analyses and carry out empirical tests to evaluate their performance. This approach allows us to ask and answer basic questions about video and other sources of temporal image sequences. Questions of interest include: How can complex streams of video be organized, archived and searched on the basis of visual content? How can multiple views of a captured scene be used to reconstruct a three-dimensional model of the scene? Key novelty comes about via an uncommon level of explicit analytic detail in system specification, which provides benefits of clarity in how the system operates and greatly decreased need for training data in comparison to most other systems.******My research contributes to Canada in three important ways. First, it advances the discipline of computer vision as we expose basic properties of image information and develop corresponding algorithms for incorporation into vision machines; this advance allows Canada to compete successfully in a critical area of science and technology. Second, it provides students with hands on experience and skills in vital areas of investigation: computer vision, video processing and, more generally, computer science and engineering. Students go on to make careers in science, technology, teaching and elsewhere, taking with them the enrichment of having been involved in research. Third, it paves the way for novel ways to exploit image data in practice. Applications of interest include video processing for everyday concerns ranging from the internet to personal handheld devices, video enhancement so that content is more readily interpretable and intelligent processing modules for robots and autonomous vehicles. The work thereby benefits both producers as well as consumers of such technologies and services.*****