Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding
Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding
复制标题
DOI：
10.15607/rss.2021.xvii.006
复制
发表时间：
2021-07
期刊：
影响因子：
--
通讯作者：
Grace Zhang;Li Zhong;Youngwoon Lee;Joseph J. Lim
中科院分区：
文献类型：
--
作者：
Grace Zhang;Li Zhong;Youngwoon Lee;Joseph J. Lim
将策略从一个环境转移到另一个环境的能力是在任务监督不可用的现实环境中高效机器人学习的有希望的途径。这可以让我们利用非常适合训练的环境，比如模拟器或实验室，来学习家庭或办公室里真实机器人的策略。为了成功，这种策略转移必须克服源和目标环境之间的视觉域差距（例如不同的照明或背景）和动态域差距（例如不同的机器人校准或建模误差）。然而，先前的策略转移方法要么不能处理大的域差距，要么一次只能处理一种类型的域差距。在本文中，我们提出了一种具有迭代“环境接地”的新颖策略转移方法IDAPT，该方法在(1)通过在目标环境域中接地源环境来直接最小化视觉和动态域差距，以及(2)在接地源环境上训练策略之间交替进行。这种迭代训练逐步地在两个环境之间对齐域，并使策略适应目标环境。一旦经过训练，策略就可以直接在目标环境中执行。运动和机器人操作任务的实证结果表明，我们的方法可以在最小的监督和与目标环境交互的情况下有效地跨视觉和动态域间隙转移策略。视频和代码可在https://clvrai.com/idapt上获得。
The ability to transfer a policy from one environment to another is a promising avenue for efficient robot learning in realistic settings where task supervision is not available. This can allow us to take advantage of environments well suited for training, such as simulators or laboratories, to learn a policy for a real robot in a home or office. To succeed, such policy transfer must overcome both the visual domain gap (e.g. different illumination or background) and the dynamics domain gap (e.g. different robot calibration or modelling error) between source and target environments. However, prior policy transfer approaches either cannot handle a large domain gap or can only address one type of domain gap at a time. In this paper, we propose a novel policy transfer method with iterative"environment grounding", IDAPT, that alternates between (1) directly minimizing both visual and dynamics domain gaps by grounding the source environment in the target environment domains, and (2) training a policy on the grounded source environment. This iterative training progressively aligns the domains between the two environments and adapts the policy to the target environment. Once trained, the policy can be directly executed on the target environment. The empirical results on locomotion and robotic manipulation tasks demonstrate that our approach can effectively transfer a policy across visual and dynamics domain gaps with minimal supervision and interaction with the target environment. Videos and code are available at https://clvrai.com/idapt .