Learning and Retrieval from Prior Data for Skill-based Imitation Learning-猫眼课题宝

DOI：

10.48550/arxiv.2210.11435

发表时间：

2022-10

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Soroush Nasiriany;Tian Gao;Ajay Mandlekar;Yuke Zhu

模仿学习为机器人学习通用行为提供了一条有前途的道路，但由于高数据监督要求和脆弱的泛化能力，传统上表现出有限的可扩展性。受多任务模仿学习最新进展的启发，我们研究了使用以前任务的先验数据来促进以稳健，数据高效的方式学习新任务。为了有效地利用先验数据，机器人必须从过去的经验中内化知识，并将这些知识应用于新的任务中。为此，我们开发了一个基于技能的模仿学习框架，该框架从先前的数据中提取时间上扩展的感觉运动技能，并随后学习调用这些学习技能的目标任务的策略。我们确定了几个关键的设计选择，显着提高性能的新任务，即表示学习目标，使更多的可预测的技能表示和检索为基础的数据增强机制，以增加监督范围的政策培训。在模拟和现实世界的操作域的集合上，我们证明了我们的方法显着优于现有的模仿学习和离线强化学习方法。视频和代码可在www.example.com上获得

Imitation learning offers a promising path for robots to learn general-purpose behaviors, but traditionally has exhibited limited scalability due to high data supervision requirements and brittle generalization. Inspired by recent advances in multi-task imitation learning, we investigate the use of prior data from previous tasks to facilitate learning novel tasks in a robust, data-efficient manner. To make effective use of the prior data, the robot must internalize knowledge from past experiences and contextualize this knowledge in novel tasks. To that end, we develop a skill-based imitation learning framework that extracts temporally extended sensorimotor skills from prior data and subsequently learns a policy for the target task that invokes these learned skills. We identify several key design choices that significantly improve performance on novel tasks, namely representation learning objectives to enable more predictable skill representations and a retrieval-based data augmentation mechanism to increase the scope of supervision for policy training. On a collection of simulated and real-world manipulation domains, we demonstrate that our method significantly outperforms existing imitation learning and offline reinforcement learning approaches. Videos and code are available at https://ut-austin-rpl.github.io/sailor

Learning and Retrieval from Prior Data for Skill-based Imitation Learning

文献摘要

被引文献

相似文献