Early Representation and Analysis of Visual Spacetime

我和我的合作者进行的研究是在计算机视觉的一般领域，试图赋予机器以视觉。从理论的角度来看，这是一项重要的努力，因为它涉及复杂信息处理中的基本问题，并可能产生与我们对自然系统(例如人类)中的视觉信息处理的理解有关的结果。从实用的角度来看，能够看东西的机器有可能以一种比目前更自然、更有用的方式与人类互动。我们对视觉信息的特殊研究提供了对图像(例如视频)时间序列的基本属性的洞察，并通过指示新的应用来扩展我们的技术能力。我们使用数学技术来分析图像，在分析的基础上开发处理算法，并进行实证测试以评估其性能。这使我们能够询问有关图像时间序列的基本问题，图像序列是关于我们周围世界的普遍信息来源。感兴趣的问题包括：如何将复杂的视频流解析为有意义的组件(例如，持久结构和移动对象)？如何使用捕获的场景的多个视图来重建场景的三维模型？这项研究在三个方面对加拿大做出了重要贡献。首先，它促进了计算机视觉学科的发展，因为我们揭示了图像信息的基本属性，并开发了相应的算法，用于整合到视觉机器中；这使加拿大能够在关键的科学和技术领域取得成功。其次，它为学生提供了在调查、计算机视觉以及更广泛的计算机科学等关键领域的实践经验和技能。学生们继续在科学、技术、教学和其他领域从事职业，同时也带来了参与研究的丰富经验。第三，它为实际利用图像数据的新方法铺平了道路。令人感兴趣的应用包括用于从互联网到个人手持设备的日常事务的视频处理，使内容更容易理解的视频增强，以及用于机器人的智能处理模块。

The research that my collaborators and I carry out is in the general area of computer vision, the attempt to endow machines with a sense of sight. From a theoretical point of view this is an important endeavor as it bears on fundamental issues in complex information processing, and may yield results that bear on our understanding of visual information processing in natural systems (e.g., humans), as well. From a practical point of view, machines that can see have the potential to interact with humans in a much more natural and useful fashion than is currently the case. Our particular studies of visual information provide insights into the fundamental properties of temporal sequences of image (e.g., video) and extend our technological capabilities by indicating novel applications. We use mathematical techniques to analyze images, develop processing algorithms based on our analyses and carry out empirical tests to evaluate their performance. This allows us to ask basic questions about temporal sequences of images, a pervasive source of information about our surround world. Questions of interest include: How can complex streams of video be parsed into meaningful components (e.g., persistent structures and moving objects)? How can multiple views of a captured scene be used to reconstruct a three-dimensional model of the scene? This research contributes to Canada in three important ways. First, it advances the discipline of computer vision as we expose basic properties of image information and develop corresponding algorithms for incorporation into vision machines; this allows Canada to compete successfully in a critical area of science and technology. Second, it provides students with hands on experience and skills in vital areas of investigation, computer vision and, more generally, computer science. Students go on to make careers in science, technology, teaching and elsewhere, taking with them the enrichment of having been involved in research. Third, it paves the way for novel ways to practically exploit image data. Applications of interest include video processing for everyday concerns ranging from the internet to personal handheld devices, video enhancement so that content is more readily interpretable and intelligent processing modules for robots.