Simultaneously Learning Transferable Symbols and Language Groundings from Perceptual Data for Instruction Following
Simultaneously Learning Transferable Symbols and Language Groundings from Perceptual Data for Instruction Following
复制标题
DOI：
10.15607/rss.2020.xvi.102
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
N. Gopalan;Eric Rosen;G. Konidaris;Stefanie Tellex
中科院分区：
文献类型：
--
作者：
N. Gopalan;Eric Rosen;G. Konidaris;Stefanie Tellex
使机器人能够像人类一样轻松地学习任务并遵循指令，对于许多现实世界的机器人应用来说非常重要。以前的方法已经应用机器学习来教授从语言到手工构建的低维符号表示的映射，使用与伴随指令配对的演示轨迹。这些符号方法导致数据有效学习。其他方法将语言直接映射到高维控制行为，这需要较少的设计工作，但数据密集型。我们建议首先从演示数据中学习符号抽象，然后将语言映射到这些学习的抽象。这些符号抽象可以用比端到端方法少得多的数据来学习，并且支持通过自然语言进行部分行为规范，因为它们允许使用传统规划器进行规划。在训练过程中，我们的方法只需要少量的演示轨迹与自然语言配对，而不需要使用模拟器，并产生一个能够规划以完成指定目标或部分计划的自然语言指令的表示。我们将我们的方法应用到两个领域，包括一个移动的机械手，其中少量的演示使机器人能够遵循导航命令，如“在走廊尽头左转”，在环境中，它还没有遇到过。
—Enabling robots to learn tasks and follow instructions as easily as humans is important for many real-world robot applications. Previous approaches have applied machine learning to teach the mapping from language to low dimensional symbolic representations constructed by hand, using demonstration trajectories paired with accompanying instructions. These symbolic methods lead to data efficient learning. Other methods map language directly to high-dimensional control behavior, which requires less design effort but is data-intensive. We propose to first learning symbolic abstractions from demonstration data and then mapping language to those learned abstractions. These symbolic abstractions can be learned with significantly less data than end-to-end approaches, and support partial behavior spec-ification via natural language since they permit planning using traditional planners. During training, our approach requires only a small number of demonstration trajectories paired with natural language—without the use of a simulator—and results in a representation capable of planning to fulfill natural language instructions specifying a goal or partial plan. We apply our approach to two domains, including a mobile manipulator, where a small number of demonstrations enable the robot to follow navigation commands like “Take left at the end of the hallway,” in environments it has not encountered before.