HYDRA: Hybrid Robot Actions for Imitation Learning
HYDRA: Hybrid Robot Actions for Imitation Learning
复制标题
DOI：
10.48550/arxiv.2306.17237
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Suneel Belkhale;Yuchen Cui;Dorsa Sadigh
中科院分区：
文献类型：
--
作者：
Suneel Belkhale;Yuchen Cui;Dorsa Sadigh
模仿学习（IL）是一种使用专家演示进行机器人学习的高效范例。然而，通过IL学习的策略在测试时会受到状态分布变化的影响，这是由于动作预测中的复合误差导致了先前看不见的状态。为策略选择一个最小化这种分布变化的动作表示在模仿学习中至关重要。以前的工作建议使用时间动作抽象来减少复合错误，但它们通常会牺牲策略灵活性或需要特定领域的知识。为了解决这些权衡，我们引入HYDRONIC，一种利用具有两个级别的动作抽象的混合动作空间的方法：稀疏的高级别路点和密集的低级别动作。HYDRONIC在测试时动态地在动作抽象之间切换，以实现对机器人的粗粒度和细粒度控制。此外，HYDRONIC采用动作重新标记来增加数据集中动作的一致性，进一步减少分布偏移。HYDRONIC在七个具有挑战性的模拟和真实的世界环境中的表现优于先前的模仿学习方法30-40%，包括真实的世界中的长期任务，如煮咖啡和烤面包。视频可以在我们的网站上找到：https://tinyurl.com/3mc6793z
Imitation Learning (IL) is a sample efficient paradigm for robot learning using expert demonstrations. However, policies learned through IL suffer from state distribution shift at test time, due to compounding errors in action prediction which lead to previously unseen states. Choosing an action representation for the policy that minimizes this distribution shift is critical in imitation learning. Prior work propose using temporal action abstractions to reduce compounding errors, but they often sacrifice policy dexterity or require domain-specific knowledge. To address these trade-offs, we introduce HYDRA, a method that leverages a hybrid action space with two levels of action abstractions: sparse high-level waypoints and dense low-level actions. HYDRA dynamically switches between action abstractions at test time to enable both coarse and fine-grained control of a robot. In addition, HYDRA employs action relabeling to increase the consistency of actions in the dataset, further reducing distribution shift. HYDRA outperforms prior imitation learning methods by 30-40% on seven challenging simulation and real world environments, involving long-horizon tasks in the real world like making coffee and toasting bread. Videos are found on our website: https://tinyurl.com/3mc6793z