RI: CAREER: Task-Oriented Model Identification for Robust Robotic Manipulation

机器人通常依靠其机械部件和周围物体的模型来可靠地执行任务。这些模型描述了物体的形状和机械性能。这些模型被用来模拟机器人可以执行的不同动作，并选择具有最佳预测结果的动作在真实环境中执行。在实践中，由于模型的不准确，预测结果往往与实际结果不同，这种差异就是所谓的现实差距。手动设计的模型本质上是不准确的。虽然这个问题在通常在封闭、结构化和受控环境中运行的工业机器人中不那么明显，但它严重限制了机器人在开放环境中的部署，在这种环境中，它们经常遇到未知或不确定模型的新对象。例如，维修店的助理机器人每天需要操作各种工具和操作新对象。该项目的目标是开发自动化和数据驱动的对象建模方法，使机器人能够在飞行中建立对象的几何和机械模型，同时谨慎地操作它们。预期的改进可能会在几个应用领域产生影响，例如在产品工程、家用机器人和救援行动中清除碎片方面需要高度灵活性的工作车间。该项目通过创建一门关于机器人学习的新课程和教科书，并发布通用对象建模工具，同时组织博物馆展览，向更广泛的受众展示自动化对象建模和操纵技术，从而培养这些潜力。此外，该项目寻求让本科生参与新泽西州立大学罗格斯大学的研究活动，该大学服务于不同的学生群体。该项目追求的方法是通过寻找模拟与现实之间的差距最小的模型来自动生成并逐渐微调物体的机械模型。具体地说，这里的目标不是识别对象的最准确模型，而是推断足以执行给定操作任务的模型。因此，自动建模过程强烈地受到给定操作任务的指导，从而避免了不必要的计算建模工作。该项目的主要技术目标是：1)为使用从数据推断的不完善模型的控制技术的性能提供理论保证。2)开发了黑盒贝叶斯优化工具，用于从有限的视觉和交互数据中推断对象的模型。3)使用可区分的3D渲染器和物理引擎开发白盒模型识别工具。4)展示在与在杂乱的环境中操纵未知物体相关的各种任务中开发的方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Robots typically rely on models of their mechanical components and of the objects in their surroundings to perform their tasks reliably. The models describe the shapes and the mechanical properties of objects. The models are used to simulate different actions that a robot can perform, and the actions with the best forecasted outcomes are selected for execution in the real environment. In practice, the forecasted outcomes are often different from the real outcomes due to the inaccuracies of the models, this difference is what is called the reality gap. Manually-designed models are inherently inaccurate. While this problem is less pronounced in industrial robots that typically operate in closed, structured and controlled environments, it severely limits the deployment of robots to open environments where they constantly encounter novel objects with unknown or uncertain models. For example, an assistant robot in a repair shop needs to manipulate various tools and operate on new objects everyday. The goal of this project is to develop automated and data-driven object modeling methods that will allow robots to build geometric and mechanical models of objects on the fly while manipulating them cautiously. Anticipated improvements have the potential for impact in several application areas, such as job shops that require high flexibility in product engineering, household robotics, and debris removal in rescue operations. This project fosters these potentials by creating a new course and textbook in robot learning, and releasing general purpose object modeling tools, while organizing museum exhibitions that will expose automated object modeling and manipulation techniques to a wider audience. Additionally, the project seeks to involve undergraduates in research activities at Rutgers, The State University of New Jersey, which serves a diverse student population.The approach pursued in this project is to automatically generate and gradually fine-tune mechanical models of objects by searching for models that minimize the gaps between simulation and reality. Specifically, the goal here is not to identify the most accurate model of an object, but rather to infer models that are sufficiently accurate to perform a given manipulation task. Therefore, the automated modeling process is strongly guided by the given manipulation task, unnecessary computational modeling efforts are thus avoided. The main technical objectives of this project are to: 1) Provide theoretical guarantees on the performance of control techniques using imperfect models inferred from data. 2) Develop black-box Bayesian optimization tools for inferring models of objects from limited vision and interaction data. 3) Develop white-box model identification tools using differentiable 3D renderers and physics engines. 4) Demonstrate the developed methods on a diverse range of tasks related to manipulating unknown objects in cluttered environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.