TRANS-AM: Transfer Learning by Aggregating Dynamics Models for Soft Robotic Assembly
TRANS-AM: Transfer Learning by Aggregating Dynamics Models for Soft Robotic Assembly
复制标题
DOI：
10.1109/icra48506.2021.9561081
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Kazutoshi Tanaka;Ryo Yonetani;Masashi Hamaya;Robert Lee;Felix von Drigalski;Yoshihisa Ijiri
中科院分区：
文献类型：
--
作者：
Kazutoshi Tanaka;Ryo Yonetani;Masashi Hamaya;Robert Lee;Felix von Drigalski;Yoshihisa Ijiri
实际的工业装配场景通常需要机器人智能体快速调整其技能以适应看不见的任务。虽然迁移强化学习（RL）可以实现这种快速适应，但许多先前的工作必须从源环境中收集许多样本，以无模型的方式学习目标任务，这在实际层面上仍然缺乏样本效率。在这项工作中，我们开发了一种新的迁移RL方法，名为聚合动力学模型的TRANSfer学习（TRANS-AM）。TRANS-AM基于基于模型的强化学习（MBRL），其高水平的采样效率，只需要从源环境中收集动力学模型。具体来说，它学习在MBRL循环中自适应地聚合源动态模型，以更好地适应目标环境的状态转换动态，并在那里执行最佳操作。作为一个案例研究，以显示所提出的方法的有效性，我们解决了一个具有挑战性的接触丰富的钉孔任务与可变孔方向使用软机器人。我们的评估与仿真和真实的机器人实验表明，TRANS-AM使软机器人完成目标任务时，从零开始学习的任务相比，更少的插曲。
Practical industrial assembly scenarios often require robotic agents to adapt their skills to unseen tasks quickly. While transfer reinforcement learning (RL) could enable such quick adaptation, much prior work has to collect many samples from source environments to learn target tasks in a model-free fashion, which still lacks sample efficiency on a practical level. In this work, we develop a novel transfer RL method named TRANSfer learning by Aggregating dynamics Models (TRANS-AM). TRANS-AM is based on model-based RL (MBRL) for its high-level sample efficiency, and only requires dynamics models to be collected from source environments. Specifically, it learns to aggregate source dynamics models adaptively in an MBRL loop to better fit the state-transition dynamics of target environments and execute optimal actions there. As a case study to show the effectiveness of this proposed approach, we address a challenging contact-rich peg-in-hole task with variable hole orientations using a soft robot. Our evaluations with both simulation and real-robot experiments demonstrate that TRANS-AM enables the soft robot to accomplish target tasks with fewer episodes compared when learning the tasks from scratch.