NRI: Robust and Efficient Physics-based Learning and Reasoning in Degraded Environments

该项目将进行基础研究，开发和集成物理驱动的推理和规划技术，使自主机器人能够操作未知的不规则对象，并在非结构化、动态环境中导航。开发的技术将部署在RoboMantis上：一种四条腿的轮式机器人，可以协助执行第一反应任务。该项目将填补现有的从数据中学习未知对象的模型研究和开发足够的机器人操作和运动仿真工具之间的重要空白，通过回答三个基本问题：1)如何有效地模拟机器人动作对具有不确定模型的对象的影响？2)如何使用物理仿真工具来规划在非结构化地形中导航的操作和运动策略？3)如何在飞行中学习物体的物理模型？该项目建立在计算机视觉、物理模拟和规划的基础上，朝着开发用于瓦砾中机器人导航的高效工具集的方向发展。该项目的主要技术目标是：1)开发物理模拟工具，可用于高效地推断刚性和非刚性物体的模型，并用于稳健规划；2)开发用于从有限的视觉和交互数据中学习物体模型的优化工具；3)开发可以利用学习的模型的操纵和导航算法；以及4)演示与搜救行动相关的各种任务的完全集成的系统，例如在杂乱中操纵未知物体和在瓦砾中导航。该项目将采用贝叶斯方法，通常在瓦砾堆中发现的物体(如碎片和岩石)的模型将从几张提供场景局部视图的RGB-D图像中推断出来，并根据它们在移动和操作动作中对机器人施加的力的反应来推断。各种模型的假设将被用来模拟施加在物体上的力的影响。最好地再现观察到的力的影响的模型将被赋予最高的概率。然后，推断出的模型将被用于规划健壮的操作和运动动作，使机器人能够清理道路并在一堆碎片中前进。该项目汇集了一个跨学科的研究团队，他们拥有计算机视觉、物理模拟和规划方面的专业知识。开发的解决方案的实施将以开放源码软件包的形式提供给研究界。这将与教育材料的生成相结合，特别是关于需要物理推理的操纵挑战的编程作业，这些作业将与学术界分享。这些材料旨在通过使用机器人技术提供的实践经验，吸引刚开始学习的本科生进入STEM，同时也让他们接触到基本的方法和数据驱动的工具。在适当的时候，将努力向K-12学生介绍研究，特别是硬件演示，以培养他们对机器人技术的早期兴趣，这涉及到STEM的各个方面。在此过程中，PIS将致力于利用罗格斯大学的多样性计划来招募和支持代表不足的群体。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project will perform fundamental research into developing and integrating physics-driven reasoning and planning techniques to enable autonomous robots to manipulate unknown irregular objects and navigate in unstructured, dynamic environments. The developed techniques will be deployed on RoboMantis: a four-legged, wheeled robot that can assist in first-response missions. The project will fill the important gap between existing research on learning models of unknown objects from data and research on developing adequate simulation tools for robotic manipulation and locomotion by answering three fundamental questions: 1) How to efficiently simulate the effects of robotic actions on objects with uncertain models? 2) How to use physics simulation tools to plan manipulation and locomotion strategies for navigating in unstructured terrains? and, 3) How to learn physical models of objects on the fly? The project builds on top of progress in computer vision, physics simulation, and planning, towards developing an efficient toolset for robotic navigation in rubble.The main technical objectives of this project are to: 1) Develop physics simulation tools that can be used for efficiently inferring models of both rigid and non-rigid objects and for robust planning, 2) Develop optimization tools for learning models of objects from limited vision and interaction data, 3) Develop manipulation and navigation algorithms that can leverage the learned models, and 4) Demonstrate the fully integrated system on a diverse range of tasks related to search and rescue operations, such as manipulating unknown objects in clutter and navigating in rubble. The project will adopt a Bayesian approach where models of objects that are typically found in piles of rubble, such as debris and rocks, will be inferred from a few RGB-D images providing partial views of the scenes, and also from their responses to forces applied by the robot during locomotion and manipulation actions. Hypotheses of various models will be used to simulate the effects of the exerted forces on the objects. Models that best reproduce the observed effects of the forces will be given the highest probabilities. The inferred models will then be used to plan robust manipulation and locomotion actions that allow the robot to clear its way and advance through a pile of debris. The project brings together an interdisciplinary team of investigators who have expertise in computer vision, physics simulation and planning. Implementations of the developed solutions will be provided to the research community as open-source software packages. This will be coupled with the generation of educational material, especially programming assignments on manipulation challenges that require physics reasoning, which will be shared with the academic community. The material will aim to attract undergraduate students early in their studies to STEM by using hands-on experience that can be provided with the use of robotics, while also exposing them to foundational methods and data-driven tools. When appropriate, efforts will be made to introduce the research, in particular the hardware demonstrations, to K-12 students to cultivate their early interests in robotics, which touches all aspects of STEM. In the process, the PIs will aim to leverage diversity programs at Rutgers University to recruit and support underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.