Language-Conditioned Imitation Learning for Robot Manipulation Tasks
Language-Conditioned Imitation Learning for Robot Manipulation Tasks
复制标题
DOI：
--
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Simon Stepputtis;Joseph Campbell;Mariano Phielipp;Stefan Lee;Chitta Baral;H. B. Amor
中科院分区：
文献类型：
--
作者：
Simon Stepputtis;Joseph Campbell;Mariano Phielipp;Stefan Lee;Chitta Baral;H. B. Amor
模仿学习是教授机器人运动技能的一种流行方法。然而，大多数方法集中于仅从执行跟踪提取策略参数（即，运动轨迹和感知数据）。人类专家和机器人之间没有足够的通信渠道来描述任务的关键方面，例如目标对象的属性或运动的预期形状。出于对人类教学过程的洞察，我们引入了一种将非结构化自然语言融入模仿学习的方法。在训练时，专家可以沿着口头描述提供演示，以便描述潜在的意图（例如，“去大的绿色碗”）。然后，训练过程将这两种模式相互关联，以编码语言，感知和运动之间的相关性。由此产生的语言调节的可视化策略可以在运行时根据新的人类命令和指令进行调节，这允许对训练的策略进行更细粒度的控制，同时还减少了情景模糊性。我们在一组模拟实验中演示了我们的方法如何学习七自由度机器人手臂的语言条件操作策略，并将结果与各种替代方法进行比较。
Imitation learning is a popular approach for teaching motor skills to robots. However, most approaches focus on extracting policy parameters from execution traces alone (i.e., motion trajectories and perceptual data). No adequate communication channel exists between the human expert and the robot to describe critical aspects of the task, such as the properties of the target object or the intended shape of the motion. Motivated by insights into the human teaching process, we introduce a method for incorporating unstructured natural language into imitation learning. At training time, the expert can provide demonstrations along with verbal descriptions in order to describe the underlying intent (e.g., "go to the large green bowl"). The training process then interrelates these two modalities to encode the correlations between language, perception, and motion. The resulting language-conditioned visuomotor policies can be conditioned at runtime on new human commands and instructions, which allows for more fine-grained control over the trained policies while also reducing situational ambiguity. We demonstrate in a set of simulation experiments how our approach can learn language-conditioned manipulation policies for a seven-degree-of-freedom robot arm and compare the results to a variety of alternative methods.