RI: CAREER: Task-Oriented Model Identification for Robust Robotic Manipulation

RI：职业：鲁棒机器人操作的面向任务的模型识别

• 批准号: 1846043
• 负责人: Abdeslam Boularias
• 金额: $ 53.59万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1846043&HistoricalAwards=false
• 关键词: RI; CAREER; Task; Oriented; Model

Robots typically rely on models of their mechanical components and of the objects in their surroundings to perform their tasks reliably. The models describe the shapes and the mechanical properties of objects. The models are used to simulate different actions that a robot can perform, and the actions with the best forecasted outcomes are selected for execution in the real environment. In practice, the forecasted outcomes are often different from the real outcomes due to the inaccuracies of the models, this difference is what is called the reality gap. Manually-designed models are inherently inaccurate. While this problem is less pronounced in industrial robots that typically operate in closed, structured and controlled environments, it severely limits the deployment of robots to open environments where they constantly encounter novel objects with unknown or uncertain models. For example, an assistant robot in a repair shop needs to manipulate various tools and operate on new objects everyday. The goal of this project is to develop automated and data-driven object modeling methods that will allow robots to build geometric and mechanical models of objects on the fly while manipulating them cautiously. Anticipated improvements have the potential for impact in several application areas, such as job shops that require high flexibility in product engineering, household robotics, and debris removal in rescue operations. This project fosters these potentials by creating a new course and textbook in robot learning, and releasing general purpose object modeling tools, while organizing museum exhibitions that will expose automated object modeling and manipulation techniques to a wider audience. Additionally, the project seeks to involve undergraduates in research activities at Rutgers, The State University of New Jersey, which serves a diverse student population.The approach pursued in this project is to automatically generate and gradually fine-tune mechanical models of objects by searching for models that minimize the gaps between simulation and reality. Specifically, the goal here is not to identify the most accurate model of an object, but rather to infer models that are sufficiently accurate to perform a given manipulation task. Therefore, the automated modeling process is strongly guided by the given manipulation task, unnecessary computational modeling efforts are thus avoided. The main technical objectives of this project are to: 1) Provide theoretical guarantees on the performance of control techniques using imperfect models inferred from data. 2) Develop black-box Bayesian optimization tools for inferring models of objects from limited vision and interaction data. 3) Develop white-box model identification tools using differentiable 3D renderers and physics engines. 4) Demonstrate the developed methods on a diverse range of tasks related to manipulating unknown objects in cluttered environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人通常依靠其机械部件和周围物体的模型来可靠地执行任务。模型描述了物体的形状和力学性能。该模型用于模拟机器人可以执行的不同动作，并选择具有最佳预测结果的动作在真实环境中执行。在实践中，由于模型的不准确性，预测结果往往与实际结果不同，这种差异被称为现实差距。手工设计的模型本质上是不准确的。虽然这个问题在通常在封闭、结构化和受控环境中运行的工业机器人中不太明显，但它严重限制了机器人在开放环境中的部署，在开放环境中，机器人不断遇到具有未知或不确定模型的新物体。例如，维修车间的助理机器人每天需要操作各种工具和操作新物体。该项目的目标是开发自动化和数据驱动的对象建模方法，使机器人能够在飞行中建立物体的几何和力学模型，同时谨慎地操作它们。预期的改进可能会对几个应用领域产生影响，例如产品工程中需要高度灵活性的作业车间、家用机器人和救援行动中的碎片清除。该项目通过创建机器人学习的新课程和教科书，发布通用对象建模工具，同时组织博物馆展览，向更广泛的受众展示自动化对象建模和操作技术，从而培养这些潜力。此外，该项目还寻求让本科生参与新泽西州罗格斯州立大学的研究活动，这所大学为不同的学生群体提供服务。在这个项目中所追求的方法是通过寻找模型来自动生成和逐渐微调物体的力学模型，使模拟和现实之间的差距最小化。具体来说，这里的目标不是确定对象的最精确模型，而是推断出足够精确的模型来执行给定的操作任务。因此，自动化建模过程由给定的操作任务强烈引导，从而避免了不必要的计算建模工作。本项目的主要技术目标是：1)利用从数据推断的不完善模型为控制技术的性能提供理论保证。2)开发黑盒贝叶斯优化工具，从有限的视觉和交互数据中推断物体模型。3)使用可微分3D渲染器和物理引擎开发白盒模型识别工具。4)在杂乱环境中操纵未知物体的各种任务中演示开发的方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Toward Fully Automated Metal Recycling using Computer Vision and Non-Prehensile Manipulation

使用计算机视觉和非预握操作实现全自动金属回收

• 发表时间: 2021
• 期刊: Proceedings of the 17th IEEE International Conference on Automation Science and Engineering
• 作者: Han, Shuai;Huang, Baichuan;Song, Changkyu;Feng, Si Wei;Xu, Ming;Boularias, Abdeslam;Yu, Jingjin
• 通讯作者: Yu, Jingjin

Interleaving Monte Carlo Tree Search and Self-Supervised Learning for Object Retrieval in Clutter

• DOI: 10.1109/icra46639.2022.9812132
• 发表时间: 2022-02
• 期刊: 2022 International Conference on Robotics and Automation (ICRA)
• 作者: Baichuan Huang;Teng Guo;Abdeslam Boularias;Jingjin Yu

Identifying Mechanical Models through Differentiable Simulations

通过可微分模拟识别机械模型

• 发表时间: 2020
• 期刊: Proceedings of Machine Learning Research
• 作者: Song, Changkyu;Boularias, Abdeslam
• 通讯作者: Boularias, Abdeslam

A Self-supervised Learning System for Object Detection in Videos Using Random Walks on Graphs

• DOI: 10.1109/icra48506.2021.9561271
• 发表时间: 2020-11
• 期刊: 2021 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: Juntao Tan;Changkyu Song;Abdeslam Boularias

Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations

从视觉演示中学习顺序操作任务的感觉运动原语

• DOI: 10.1109/icra46639.2022.9811703
• 发表时间: 2022
• 期刊: International Conference on Robotics and Automation (ICRA
• 作者: Liang, Junchi;Wen, Bowen;Bekris, Kostas;Boularias, Abdeslam
• 通讯作者: Boularias, Abdeslam