Collaborative Research: RI: Medium: Bridging the Semantic-Metric Gap via Multinocular Image Integration

合作研究：RI：Medium：通过多目图像集成弥合语义度量差距

• 批准号: 2312747
• 负责人: Ahmad Ahmad
• 金额: $ 7.5万
• 依托单位: University of Tennessee Knoxville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2312747&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Medium; Bridging

Humans and other animals can effortlessly and subconsciously reconstruct the 3D world around them from the video imagery streaming to their eyes, and successfully use it for navigation, food-finding, predator avoidance, etc. Computer vision 3D technology has been evolving rapidly to reconstruct the world from a set of cameras and locate these cameras in the environment. This technology is a basis of navigation as in automated driving, robot navigation, and drone flights; a basis of manipulation as in robotic manufacturing, robotic medical interventions, etc.; measurement in metrology; modeling for the entertainment industry; and a host of other applications. As a result, 3D vision has experienced an exponential growth in capability, efficiency, and robustness. Despite this phenomenal growth arising from exploiting what is currently achievable, fundamental shortcomings exist that need to be addressed to enlarge the scope of application and to increase robustness in existing ones. First, images from rapidly moving cameras (e.g., drones and pedestrians) are often blurry and lack features; indoor scenes and others which have textureless surfaces or surfaces with repeated texture lack features or have indistinguishable features; and there are other examples which are often beyond the capabilities of current technologies. Second, image sensing typically enjoys a high degree of redundancy which is often discarded in current algorithms, thus forfeiting the opportunity to use the high information content inherent in the redundancy. Third, there is often a large gap between the internal representations used in the current technology, which are often point-based, and a semantic representation of the scene, which are more resonant with an understanding of underlying curves (e.g., ridges) and surface patches (faces) of an object. This project aims to remedy these shortcomings.Several technical challenges need to be addressed to achieve these goals. First, this project identifies that the notion of numerical stability, currently confounded with degeneracy, should be thoroughly studied and analyzed for key multiview geometry (MVG) tasks. The stability requirement leads to a new class of techniques which will be implemented and made readily available to the community to help avoid failure modes in a broad selection of MVG problems. Second, the development of tools to solve very large polynomial systems is an enabling technology that will transform not just multiview geometry problems, but also a broad range problem from other scientific areas. Third, these developments will enable a novel MVG approach based on curves, surfaces, and their differential geometry for relative pose estimation, absolute pose estimation, and 3D reconstruction. This will serve to bridge the semantic-metric gap that exists between geometrically accurate 3D point clouds/meshes and semantically meaningful organizations in terms of objects, object parts, spatial layout, mapping, etc. In conjunction, these three streams of research will allow direct, efficient and reliable integration of information across a large number of views in multinocular vision systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类和其他动物可以轻松，潜意识地重建周围的3D世界，从视频图像流到他们的眼睛，并成功地将其用于导航，食物调查，避免食物等。计算机视觉3D技术正在迅速发展，从一组相机中重建了世界，并在环境中定位了这些镜头。这项技术是导航的基础，例如自动驾驶，机器人导航和无人机航班；机器人制造，机器人医疗干预等中的操作基础；计量学的测量；娱乐业建模；以及许多其他应用程序。结果，3D视觉在能力，效率和鲁棒性方面呈指数增长。尽管利用当前成功的事物而产生了这种惊人的增长，但存在基本的缺点，需要解决，以提高应用程序的范围并提高现有的应用范围。首先，来自快速移动的相机（例如，无人机和行人）的图像通常是模糊的，缺乏特征。室内场景和其他具有无纹理表面或表面具有重复纹理的表面的其他场景缺乏特征或具有无法区分的功能；而且还有其他示例通常超出当前技术的能力。其次，图像传感通常享有高度的冗余，通常在当前算法中丢弃，从而丧失了使用冗余固有的高信息内容的机会。第三，当前技术中使用的内部表示之间通常存在很大的差距，而这些内部表示通常是基于点的，并且对场景的语义表示，这更加共鸣，对对象的潜在曲线（例如山脊）和表面斑块（例如面孔）的理解更加共鸣。该项目旨在记住这些缺点。需要解决几个技术挑战以实现这些目标。首先，该项目确定了当前与退化性混淆的数值稳定性的概念应进行彻底研究并分析关键的多浏览几何（MVG）任务。稳定性需求会导致一类新的技术，这些技术将在社区中易于实施，并可以在各种MVG问题中避免故障模式。其次，开发解决非常大的多项式系统的工具是一种有利的技术，它不仅会改变多视图几何问题，而且会从其他科学领域转变出广泛的范围问题。第三，这些发展将基于曲线，表面及其差异几何形状来实现一种新型的MVG方法，以进行相对姿势估计，绝对姿势估计和3D重建。这将有助于弥合几何准确的3D点云/网眼和语义上有意义的组织之间存在的语义差距，从对象，对象部分，空间布局，映射等方面，这三个研究流将允许直接，效率和可靠的统计信息整体统计信息，这三个研究将允许多个统计信息的统计信息范围内的信息范围内的信息。使用基金会的智力优点和更广泛的影响标准，认为通过评估被认为是宝贵的支持。