NRI: INT: A New Paradigm for Geometric Reasoning through Structure from Category

NRI：INT：通过类别结构进行几何推理的新范式

• 批准号: 1925281
• 负责人: Laszlo Jeni
• 金额: $ 47.37万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1925281&HistoricalAwards=false
• 关键词: NRI; INT; New; Paradigm; Geometric

The task of a robot determining the 3D shape and pose of an object is critical to the advancement of generally deployed collaborative robotics. Current artificial intelligence (AI) still struggles with such tasks. Until now the problem of inferring an object's 3D pose and shape (i.e., geometric reasoning) through AI has largely been treated as a 3D supervised learning problem. That is, the AI is given 2D images with corresponding 3D labels to learn. These 3D labels are costly and error prone to obtain at a large scale, acting as an intrinsic barrier to the advancement of geometric reasoning within robotics. In this project, the research team advocates a new paradigm for geometric reasoning that requires no 3D supervision - using a mathematical framework called "structure from category". Success will result in autonomous systems such as vehicles, robots, and drones with dramatically enhanced perception and planning abilities to navigate their way in 3D world. Deep neural networks (DNNs) - the heart of most AI systems in robotics - are currently used like black boxes. It is hard to instill them explicitly with real-world knowledge; instead they learn implicitly through labelled examples. This becomes a problem when such examples are not available in abundance. This research aims to shift this paradigm for AI from an opaque black-box to a transparent "glass-box", to facilitate the principled inclusion of explicit constraints and priors such as geometry. In this project, the research team has a novel way around this impasse, by reinterpreting the nonlinearities within a DNN as hierarchical sparsity constraints. Using this approach, the research team advocates for a method of inferring 3D shape and pose solely from 2D labels, dramatically improving the ability of modern DNNs to perform effective geometric reasoning at scale across large amounts of data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人确定物体的三维形状和姿态的任务对于普遍部署的协作机器人的进步至关重要。目前的人工智能（AI）仍在努力完成这些任务。到目前为止，通过人工智能推断物体的3D姿态和形状（即几何推理）的问题在很大程度上被视为3D监督学习问题。即给予AI带有相应3D标签的2D图像进行学习。这些3D标签成本高昂，而且在大规模获取时容易出错，这是机器人技术中几何推理进步的内在障碍。在这个项目中，研究团队提倡一种不需要3D监督的几何推理新范式——使用一种称为“类别结构”的数学框架。成功将导致自动系统，如车辆、机器人和无人机，具有显著增强的感知和规划能力，可以在3D世界中导航。深度神经网络（dnn）是机器人技术中大多数人工智能系统的核心，目前被用作黑匣子。很难明确地向他们灌输现实世界的知识；相反，他们通过标记的例子含蓄地学习。当这样的例子不充足时，这就成了一个问题。这项研究旨在将人工智能的这种范式从不透明的黑盒转变为透明的“玻璃盒”，以促进明确约束和先验（如几何）的原则性包含。在这个项目中，研究团队通过将深度神经网络中的非线性重新解释为层次稀疏性约束，找到了一种解决这个僵局的新方法。使用这种方法，研究团队提倡一种仅从2D标签推断3D形状和姿势的方法，极大地提高了现代dnn在大量数据中执行有效几何推理的能力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

• 发表时间: 2020-10
• 期刊: ArXiv
• 作者: Chen-Hsuan Lin;Chaoyang Wang;S. Lucey

MBW: Multi-view Bootstrapping in the Wild

MBW：野外多视图引导

• 发表时间: 2022
• 期刊: Advances in neural information processing systems
• 作者: Mosam Dabhi;Chaoyang Wang;Tim Clifford;László Jeni;Ian Fasel;Simon Lucey
• 通讯作者: Simon Lucey

Deep Non-Rigid Structure From Motion With Missing Data

• DOI: 10.1109/tpami.2020.2997026
• 发表时间: 2019-07
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Chen Kong;S. Lucey

Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild

• DOI: 10.1109/3dv50981.2020.00011
• 发表时间: 2020-11
• 期刊: 2020 International Conference on 3D Vision (3DV)
• 作者: Chaoyang Wang;Chen-Hsuan Lin;S. Lucey

High Fidelity 3D Reconstructions with Limited Physical Views

• DOI: 10.1109/3dv53792.2021.00137
• 发表时间: 2021-10
• 期刊: 2021 International Conference on 3D Vision (3DV)
• 作者: Mosam Dabhi;Chaoyang Wang;Kunal Saluja;László A. Jeni;Ian R. Fasel;S. Lucey