NRI: Collaborative Research: Experiential Learning for Robots: From Physics to Actions to Tasks

NRI：协作研究：机器人的体验式学习：从物理到动作再到任务

• 批准号: 1637479
• 负责人: Dieter Fox
• 金额: $ 75.2万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2020-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1637479&HistoricalAwards=false
• 关键词: NRI; Collaborative; Research; Experiential; Learning

Recent advances in machine learning coupled with unprecedented archives of labeled data are advancing machine perception at a remarkable rate. However, applying these advances to robotics has not advanced as quickly because learning for robotics requires both active interaction with the physical world, and the ability to generalize over a variety of task contexts. This project addresses this knowledge gap through the development of new learning methods to produce experience-based models of physics. In this approach, an object or category specific model of physics is learned directly from perceptual data rather than deploying general-purpose physical simulation methods. These physical models will support both direct control of action - for example pouring a liquid into a container, and the learning of the physical effects of sequences of actions - for example planning to handle fluids in a laboratory. More generally, these methods will provide a means for robots to learn how to handle fluids, soft materials, and other complex physical phenomena.The proposed experiential learning framework will build on recent advances in deep neural networks. The key problem is to learn the mappings between raw perceptual and control data via a low-dimensional implicit physics space representing a perception-based physical model of how an object acts in the environment. Three directions will be investigated: 1) the development of experiential physics models for object interaction and fluid flow that have strong predictive capabilities, 2) creating mappings directly from experiential models to control of actions such as pouring or moving an object, 3) the assembly of local experience-based controllers into complex tasks from interactive demonstration. Additionally, the project will develop unique data sets that include physical models, simulations, data components, and learned components that other groups can access and build on to enable comparative research similar to what has emerged in machine perception.

机器学习的最新进展加上前所未有的标记数据档案正在以惊人的速度推进机器感知。然而，将这些进步应用于机器人技术的进展并不快，因为机器人技术的学习既需要与物理世界的积极互动，也需要在各种任务环境中进行概括的能力。本项目通过开发新的学习方法来建立基于经验的物理模型，解决了这一知识差距。在这种方法中，直接从感知数据中学习对象或类别特定的物理模型，而不是部署通用的物理模拟方法。这些物理模型将支持对动作的直接控制——例如将液体倒入容器中，以及对动作序列的物理效应的学习——例如计划在实验室中处理液体。更一般地说，这些方法将为机器人学习如何处理流体、软材料和其他复杂的物理现象提供一种手段。提出的体验式学习框架将建立在深度神经网络的最新进展之上。关键问题是通过低维隐式物理空间来学习原始感知和控制数据之间的映射，该物理空间表示对象在环境中如何行为的基于感知的物理模型。将研究三个方向：1)开发具有强大预测能力的物体交互和流体流动的体验物理模型；2)直接从体验模型创建映射到诸如倾倒或移动物体等动作的控制；3)从交互式演示中将基于本地经验的控制器组装到复杂任务中。此外，该项目将开发独特的数据集，包括物理模型、模拟、数据组件和学习组件，其他团队可以访问和构建这些数据集，以实现类似于机器感知中出现的比较研究。

项目成果

Robot Object Referencing through Legible Situated Projections

通过清晰的定位投影来引用机器人对象

• DOI: 10.1109/icra.2019.8793638
• 发表时间: 2019
• 期刊: International Conference on Robotics and Automation
• 作者: Weng, Thomas;Perlmutter, Leah;Nikolaidis, Stefanos;Srinivasa, Siddhartha;Cakmak, Maya
• 通讯作者: Cakmak, Maya

Synthesizing Robot Manipulation Programs from a Single Observed Human Demonstration

从单个观察到的人类演示中合成机器人操作程序

• 发表时间: 2019
• 期刊: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Huang, J.;Fox, D.;Cakmak, M.
• 通讯作者: Cakmak, M.