NRI: INT: COLLAB: Integrated Modeling and Learning for Robust Grasping and Dexterous Manipulation with Adaptive Hands

NRI：INT：COLLAB：利用自适应手实现稳健抓取和灵巧操作的集成建模和学习

• 批准号: 1734190
• 负责人: Aaron Dollar
• 金额: $ 63.25万
• 依托单位: Yale University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1734190&HistoricalAwards=false
• 关键词: NRI; INT; COLLAB; Integrated; Modeling

Robots need to effectively interact with a large variety of objectsthat appear in warehouses and factories as well as homes and offices.This requires robust grasping and dexterous manipulation of everydayobjects through low cost robots and low complexity solutions.Traditionally, robots use rigid hands and analytical models for suchtasks, which often fail in the presence of even small errors. Newcompliant hands promise improved performance, while minimizingcomplexity, and increased robustness. Nevertheless, they areinherently difficult to sense and model. This project combines ideasfrom different robotics sub-fields to address this limitation. Itutilizes progress in machine learning and builds on a strong traditionin robot modeling. The objective is to provide adaptive, compliantrobots that are better in grasping objects in the presence of multipleunknown contact points and sliding or rolling objects in-hand. Thebroader impact will be strengthened by the open release of new ormodified robot hand designs, improved control algorithms and software,as well as corresponding data sets. Furthermore, academicdissemination will be accompanied by educational outreach toundergraduate and high school students.Towards the above objective, the first step will be the definition ofnew hybrid models appropriate for adaptive, compliant hands. Thiswill happen by improving analytical solutions and extending them toallow adaptation based on data via novel, time-efficient learningmethods. The objective is to capture model uncertainty inherent inreal-world interactions; a process that suffers from data scarcity.In order to reduce the amount of data required for learning, differentmodels will be tailored to specific tasks through an automateddiscovery of these tasks and of underlying motion primitives for eachone of them. This task identification process will operate iterativelywith learning and utilize improved models to discover new tasks. Itcan also provide feedback for improved hand design. Once theselearning-based and task-focused models are available, they will beused to learn and synthesize controllers for grasping and in-handmanipulation. To learn controllers, this work will consider amodel-based, reinforcement learning approach, which will be evaluatedagainst alternatives. For controller synthesis, existing tools forthis purpose will be integrated with task planning primitives andextended through learning processes to identify the preconditionsunder which different controllers can be chained together. The projectinvolves extensive evaluation on a variety of novel adaptive hands androbotic arms designed in the PIs' labs. Modern vision-based solutionswill be used to track grasped objects and provide feedback forlearning and closed-loop control. The evaluation will measure whetherthe developed hybrid models can significantly improve robustness ofgrasping and the effectiveness of dexterous manipulation.

机器人需要与仓库、工厂、家庭和办公室中出现的各种物体进行有效的交互。这需要通过低成本的机器人和低复杂性的解决方案对日常物体进行强大的抓取和灵巧的操纵。传统上，机器人使用刚性的手和分析模型来完成此类任务，即使存在很小的错误，这些任务也常常会失败。新的顺应指针有望提高性能，同时最大限度地降低复杂性并提高稳健性。然而，它们本质上很难感知和建模。该项目结合了不同机器人子领域的想法来解决这一限制。它利用了机器学习的进步，并建立在机器人建模的强大传统之上。目标是提供自适应、顺从的机器人，在存在多个未知接触点以及手中滑动或滚动物体的情况下，能够更好地抓取物体。通过公开发布新的或改进的机器人手设计、改进的控制算法和软件以及相应的数据集，将加强更广泛的影响。此外，学术传播还将伴随着对本科生和高中生的教育推广。为了实现上述目标，第一步将是定义适合适应性、顺应性双手的新混合模型。 这将通过改进分析解决方案并扩展它们以允许通过新颖、高效的学习方法基于数​​据进行适应来实现。目标是捕捉现实世界交互中固有的模型不确定性；这是一个遭受数据稀缺的过程。为了减少学习所需的数据量，通过自动发现这些任务以及每个任务的底层运动原语，将针对特定任务定制不同的模型。该任务识别过程将通过学习迭代进行，并利用改进的模型来发现新任务。它还可以为改进手部设计提供反馈。一旦这些基于学习和以任务为中心的模型可用，它们将用于学习和合成用于抓取和手动操作的控制器。为了学习控制器，这项工作将考虑基于模型的强化学习方法，该方法将针对替代方案进行评估。对于控制器合成，用于此目的的现有工具将与任务规划原语集成，并通过学习过程进行扩展，以确定不同控制器可以链接在一起的先决条件。该项目涉及对 PI 实验室设计的各种新型自适应手和机械臂进行广泛评估。现代基于视觉的解决方案将用于跟踪抓取的物体并为学习和闭环控制提供反馈。 评估将衡量开发的混合模型是否能够显着提高抓取的鲁棒性和灵巧操作的有效性。

项目成果

Hand–object configuration estimation using particle filters for dexterous in-hand manipulation

使用粒子滤波器进行手部物体配置估计，以实现灵巧的手部操作

• DOI: 10.1177/0278364919883343
• 发表时间: 2019
• 期刊: The International Journal of Robotics Research
• 作者: Hang, Kaiyu;Bircher, Walter G.;Morgan, Andrew S.;Dollar, Aaron M.
• 通讯作者: Dollar, Aaron M.

Learning Modes of Within-Hand Manipulation

• DOI: 10.1109/icra.2018.8461187
• 发表时间: 2018-05
• 期刊: 2018 IEEE International Conference on Robotics and Automation (ICRA)
• 作者: B. Çalli;K. Srinivasan;A. S. Morgan;A. Dollar

Towards Generalized Manipulation Learning Through Grasp Mechanics-Based Features and Self-Supervision

通过基于抓取力学的特征和自我监督实现广义操纵学习

• DOI: 10.1109/tro.2021.3057802
• 发表时间: 2021
• 期刊: IEEE Transactions on Robotics
• 影响因子: 7.8
• 作者: Morgan, Andrew S.;Bircher, Walter G.;Dollar, Aaron M.
• 通讯作者: Dollar, Aaron M.

Manipulation for self-Identification, and self-Identification for better manipulation

• DOI: 10.1126/scirobotics.abe1321
• 发表时间: 2021-05-26
• 期刊: SCIENCE ROBOTICS
• 影响因子: 25
• 作者: Hang, Kaiyu;Bircher, Walter G.;Dollar, Aaron M.
• 通讯作者: Dollar, Aaron M.

Using a Variable-Friction Robot Hand to Determine Proprioceptive Features for Object Classification During Within-Hand-Manipulation

使用可变摩擦机器人手确定手内操作过程中物体分类的本体感觉特征

• DOI: 10.1109/toh.2019.2958669
• 发表时间: 2020
• 期刊: IEEE Transactions on Haptics
• 影响因子: 2.9
• 作者: Spiers, Adam J.;Morgan, Andrew S.;Srinivasan, Krishnan;Calli, Berk;Dollar, Aaron M.
• 通讯作者: Dollar, Aaron M.