Hierarchical Representation and Analysis of Visual Spacetime

基于视觉的系统是计算、机器人和人工智能领域的一个中心研究课题。这种突出表现是由于汽车、消费者和家庭技术的当前趋势（例如，Amazon Echo、Google Home、Apple HomePod）正在利用视频数据来理解环境并做出反应。此外，当我们试图了解如何从仅仅捕获的视频中恢复有关世界的详细信息时，有趣的科学问题出现了。我的研究目标是视频理解的基础理论和工程原理。我对视觉信息的特殊研究提供了对图像时间序列基本属性的见解（例如，视频），并通过指出新的应用扩展我们的技术能力。我使用数学技术来分析图像，根据分析结果开发处理算法，并进行实证测试来评估其性能。这种方法允许我们询问和回答有关视频和其他时间图像序列源的基本问题。感兴趣的问题包括：如何组织，存档和搜索的基础上的视觉内容的视频复杂的流？如何使用捕获场景的多个视图来重建场景的三维模型？关键的新奇来自于系统规范中不寻常的明确分析细节，这提供了系统如何操作的清晰度，并且与大多数其他系统相比，大大减少了对训练数据的需求。我的研究在三个重要方面为加拿大做出了贡献。首先，它推进了计算机视觉学科，因为我们揭示了图像信息的基本属性，并开发了相应的算法，以纳入视觉机器;这一进步使加拿大能够在科学和技术的关键领域成功竞争。其次，它为学生提供了调查的重要领域的实践经验和技能：计算机视觉，视频处理，更普遍的是，计算机科学和工程。学生继续在科学，技术，教学和其他地方的职业生涯，与他们一起参与研究的丰富。第三，它为在实践中利用图像数据的新方法铺平了道路。感兴趣的应用包括从互联网到个人手持设备的日常关注的视频处理，视频增强，以便内容更容易解释，以及机器人和自动驾驶汽车的智能处理模块。因此，这项工作既有利于这些技术和服务的生产者，也有利于消费者。

Vision-based systems are a central research topic in computing, robotics and artificial intelligence. This prominence arises as current trends in automotive, consumer and home technology (e.g., Amazon Echo, Google Home, Apple HomePod) are leveraging video data to understand and react to the environment. Moreover, interesting scientific questions arise as we seek to understand how it is possible to recover detailed information about the world from a mere captured video. My research targets the underpinning theoretical and engineering principles of video understanding.******My particular studies of visual information provide insights into the fundamental properties of temporal sequences of images (e.g., video) and extend our technological capabilities by indicating novel applications. I use mathematical techniques to analyze images, develop processing algorithms based on the resulting analyses and carry out empirical tests to evaluate their performance. This approach allows us to ask and answer basic questions about video and other sources of temporal image sequences. Questions of interest include: How can complex streams of video be organized, archived and searched on the basis of visual content? How can multiple views of a captured scene be used to reconstruct a three-dimensional model of the scene? Key novelty comes about via an uncommon level of explicit analytic detail in system specification, which provides benefits of clarity in how the system operates and greatly decreased need for training data in comparison to most other systems.******My research contributes to Canada in three important ways. First, it advances the discipline of computer vision as we expose basic properties of image information and develop corresponding algorithms for incorporation into vision machines; this advance allows Canada to compete successfully in a critical area of science and technology. Second, it provides students with hands on experience and skills in vital areas of investigation: computer vision, video processing and, more generally, computer science and engineering. Students go on to make careers in science, technology, teaching and elsewhere, taking with them the enrichment of having been involved in research. Third, it paves the way for novel ways to exploit image data in practice. Applications of interest include video processing for everyday concerns ranging from the internet to personal handheld devices, video enhancement so that content is more readily interpretable and intelligent processing modules for robots and autonomous vehicles. The work thereby benefits both producers as well as consumers of such technologies and services.*****