RI: Small: A Differential Geometry Paradigm for Constructing a Semantic Mid-Level Representation for Multinocular Pose Estimation and Reconstruction

3D视觉是根据从多个相机或在整个场景中移动的单个相机获得的图像构建场景的过程。然而，目前的3D视觉遭受一些问题，由于三个核心问题：强调点，而不是曲线和曲面;次优的整合信息从多个图像，并需要更丰富的语义几何结构的场景。该项目旨在通过开发新技术来纠正所有这三个缺点，这些新技术使用曲线和曲面，同时集成来自多个摄像机的信息，并使用语义原语来描述场景中的对象。这些发展将补充和提出了一个宝贵的除了现有的技术过多的三维姿态估计和重建，特别是在无纹理的场景，如人造环境。该研究的应用领域包括机器人、娱乐业、考古学、建筑、城市建模和计量学。本项目开发了一种端到端的技术，用于使用曲线和曲面的微分几何进行姿态估计和场景重建。首先，它开发了在姿态估计过程中使用点切线而不仅仅是点所需的技术，在使用已经可用的信息减少必要对应的数量方面具有变革性。它还将确定这方面的最小问题。其次，该方法将允许使用直观定义的几何方程来整合来自许多相机的信息，这些几何方程可以很容易地扩展到三个相机之外，这在视觉里程计等应用中很重要。第三，它开发了一种从3D曲线图像脚手架一样进行表面重建的技术，同时还考虑到纹理区域中的无组织点重建和无纹理区域中的一组新的微分光度约束。第四，它将基于图像的分组过程与同时进行的3D重建过程相结合，从而产生一个中间级表示，作为一组拓扑连接的曲线片段和表面片段，更好地满足语义和功能任务的要求。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

3D vision is the process of constructing a scene from images obtained from multiple cameras or a single camera that is moving throughout the scene. However, current 3D vision suffers some problems due to three core issues: the emphasis on points rather than on curves and surfaces; sub-optimal integration of information from multiple images, and the need for richer semantic geometrical structuring of the scene. This project aims to rectify all three shortcomings by developing new technologies which work with curves and surfaces, integrate information from multiple cameras simultaneously, and use semantic primitives to describes objects in the scene. These developments will complement and present an invaluable addition to the plethora of existing techniques for 3D pose estimation and reconstruction, especially in textureless scenes such as man-made environments. Applications of this research include robotics, entertainment industry, archaeology, architecture, urban modeling, and metrology.This project develops an end-to-end technology for pose estimation and scene reconstruction using differential geometry of curves and surfaces. First, it develops the technology necessary to use point-tangents as opposed to just points in the pose estimation process, transformative in reducing the number of necessary correspondences using information that is already available. It will also identify the minimal problems in this area. Second, the approach would allow for the integration of information from many cameras using intuitively defined geometric equations that can easily extend beyond three cameras, important in applications like visual odometry. Third, it develops a technology for surface reconstruction from a 3D curve graph acting like scaffold, while also taking into account both unorganized point reconstructions in textured areas and a novel set of differential photometric constraints in textureless areas. Fourth, it integrates the image-based grouping process with a simultaneous 3D reconstruction process resulting in a mid-level representation as a topologically connected set of curve fragments and surface fragments, which better matches the requirement of semantic and functional tasks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.