EAGER: Robust Data-Driven Robotic Manipulation via Bayesian Inference and Passivity-Based Control

机器人通常通过牢牢抓住物体来移动物体。有些任务不能用这种方式完成，因为物体可能很精致，或者相对于机器人的手臂或手来说很大。例如，我们在移动合上的书时使用紧握，但在翻页时使用微妙的手指运动。由于机器人的“手”可以相对于对象移动，因此机器人、对象和环境之间的接触类型在操作过程中可能会发生变化。当接触条件不同时，施加在对象上的力会产生不同的运动。相反，不同的动作可能会导致未来不同的接触。规划计算方法可以识别能够完成任务的正确的力序列和接触条件。在计划的动议执行过程中突然出现的小错误通常会通过采取纠正措施来减少。然而，这些纠正措施通常不考虑更改=联系人，相反，由于意外接触而导致的错误更为严重，最终导致任务失败。这个早期概念探索性研究资助(AGER)项目将研究创建机器人运动计划的技术，以减轻而不是放大执行此类任务期间的错误。这种涉及重大接触事件的操作任务可以在机器人应用中找到，比如装载洗碗机，从杂乱的橱柜中取出难以触及的物体，或者移动家具。该项目团队将研究新的数据驱动方法，以训练稳健的运动控制器，这些控制器是从贝叶斯神经网络派生出来的，具有特殊的结构，了解机器人学和控制原理。考虑到任务的接触丰富的性质，网络将由混合的专家组成，其中每个专家要么是控制器，要么是用于推导基于无源的控制器的存储函数。门控网络根据网络的输入选择要使用的控制器。贝叶斯网络将在给定输入的情况下提供运动命令的分布，从而允许运动控制器考虑不确定性。该项目将分三个重叠的阶段进行：调查人员将使用正式验证的工具来合成控制器，这些控制器可以证明是局部稳定的接触丰富的运动计划，并使用这些控制器来使用知识蒸馏来初始化贝叶斯神经网络权重的先验分布。这个初始化的网络将使用来自可微模拟器的数据以端到端的方式根据基于任务的奖励进行训练，其中机器人-对象-环境系统参数是不确定的。训练过的网络将在实验中进行测试，实验涉及机器人手臂推动一个大盒子越过阶梯式障碍物，这些障碍物旨在要求在操作过程中改变接触条件。该项目如果成功，将确定一种控制器合成范例，同时克服模拟与现实之间的差距和困扰纯数据驱动方法的数据低效问题，以实现接触丰富的对象操作。该项目还将推进计算控制器合成方面的知识，并为随机系统的GPU加速模拟提供新的工具。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Robots usually move objects by firmly holding on to them. Some tasks cannot be done this way, because the object may be delicate, or large relative to the robot's arm or hand. For example, we use firm holds when moving a closed book, but delicate finger motions when turning a page. Since the robot's "hand" may move relative to the object, the contact type between the robot, object, and environment can change during manipulation. Forces applied on the object create different motions when contact conditions are different. Conversely, different motions may lead to different contacts in the future. Planning computational methods can identify the right sequence of forces and contact conditions that could complete a task. Small errors that crop up during execution of planned motions would normally be reduced by taking corrective actions. However, these corrective actions often do not account for changing = contacts, and the errors instead are more critical due to unanticipated contacts, ultimately leading to failure on tasks. This EArly-concept Grant for Exploratory Research (EAGER) project will study techniques to create plans for robot motion that mitigate instead of amplify errors during execution of such tasks. Such manipulation tasks involving significant contact events can be found in robotic applications such as loading dishwashers, fetching hard-to-reach objects from cluttered cupboards, or moving furniture. The project team will study new data-driven methods to train robust motion controllers that are derived from Bayesian neural networks with special structure informed by robotics and control principles. To account for the contact-rich nature of the task, the network will consist of a mixture-of-experts, where each expert is either a controller or a storage function used to derive a passivity-based controller. A gating network chooses which controller to use given the input to the network. Bayesian networks will provide a distribution over motor commands given an input, allowing the motion controller to account for uncertainty. The project will proceed in three overlapping stages: The investigators will use tools from formal verification to synthesize controllers that provably locally stabilize contact-rich motion plans, and use these controllers to initialize a prior distribution for the weights of the Bayesian neural network using knowledge distillation. This initialized network will be trained from task-based rewards in an end-to-end manner using data from differentiable simulators, where the robot-object-environment system parameters are uncertain. The trained network will be tested in experiments involving a robot arm pushing a large box over step-like obstacles designed to require changes in contact conditions during manipulation. The project, if successful, will identify a controller synthesis paradigm that simultaneously overcomes the simulation-to-reality gap and the data-inefficiency plaguing purely data-driven approaches for contact-rich object manipulation. This project will also advance knowledge in scaling up computational controller synthesis, and contribute new tools for GPU-accelerated simulation of stochastic systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.