RI: Small: Learning to Discover Structure for 3D Vision

人们能够自发地感知图像中的结构，即有序、规则或连贯的模式和关系。当走在城市街道上时，人类可以立即识别平行线、矩形、旋转对称、重复图案和许多其他类型的结构。该项目利用海量数据的可用性和机器学习技术的最新进展，为计算机视觉中的结构发现开发了一种新的数据驱动框架。该项目开发的技术可以广泛应用于现实世界的应用，如人造环境的3D重建、虚拟和增强现实以及室内救援机器人。此外，像人类一样理解3D感知组织的能力可以使其他领域受益，包括(i)认知科学，因为它产生了新的计算模型，可用于测试和发展现有理论，并探索大脑的新细节和方面，（ii）人机交互，因为它使机器人能够根据几何形状，物理和动力学进行推理，以及（iii）建筑工程。因为它促进了与现有建筑和管理标准的互动。该项目建立在结构的正式定义之上，包括(i)组成模式，（ii）模式的复制或延续域，以及（iii）模式和域在空间和时间上的任何变化（Witkin和Tenenbaum， 1983）。这项研究有三个目的。第一个目标是通过创新机器学习方法来分别检测结构的域及其组成模式，从而奠定计算基础。第二个目标是进一步建立一个统一的结构发现框架，超越自下而上的方案和顺序处理。在这里，挑战在于结构通常跨越很大的空间范围，并且通常受到模式变形和域畸变的影响。在最后一个目标中，研究人员将该结构结合到复杂的视觉系统中，以展示其在现实应用中的优势。最终，该项目致力于显著提高3D视觉系统的有效性和效率，并丰富通用计算机视觉原理。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

People are able to spontaneously perceive structure, that is, orderly, regular, or coherent patterns and relationships, in images. When walking along a city street, a human can instantly identify parallel lines, rectangles, rotational symmetries, repetitive patterns, and many other types of structure. This project develops a novel data-driven framework for structure discovery in computer vision, leveraging the availability of massive data and recent advances in machine learning techniques. The techniques developed in this project can be applied to a wide spectrum of real-world applications such as 3D reconstruction of man-made environments, virtual and augmented reality, and indoor rescue robots. Further, the ability to understand 3D perceptual organization as humans do can benefit other fields including (i) cognitive science, as it produces new computational models which can be used to test and develop existing theories, and to explore new details and aspects of the brain, (ii) human-robot interaction, as it enables robots to reason in terms of geometric shape, physics, and dynamics, and (iii) architectural engineering, as it facilitates interactions with existing standards for construction and management of buildings.The project is built upon a formal definition of structure, consisting of (i) the constituent patterns, (ii) the domain of replication or continuation of the patterns, and (iii) any change of the pattern and domain over space and time (Witkin and Tenenbaum, 1983). The research has three aims. The first aim lays the computational foundation by innovating machine learning methods to detect the domain of the structure and its constituent patterns, respectively. The second aim further establishes a unified framework for structure discovery, going beyond the bottom-up scheme and sequential processing. Here, the challenge lies in that structure often spans large spatial extent, and is typically subject to pattern deformation and domain distortion. In the last aim, the researchers incorporate the structure in complex vision systems to demonstrate its advantage in real-world applications. Ultimately, the project strives to significantly improve the effectiveness and efficiency of 3D vision systems, and to enrich the general computer vision principles.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.