Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog
Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog
复制标题
DOI：
10.1613/jair.1.11485
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Jesse Thomason;Aishwarya Padmakumar;Jivko Sinapov;Nick Walker;Yuqian Jiang;Harel Yedidsion;Justin W. Hart;P. Stone;R. Mooney
中科院分区：
文献类型：
--
作者：
Jesse Thomason;Aishwarya Padmakumar;Jivko Sinapov;Nick Walker;Yuqian Jiang;Harel Yedidsion;Justin W. Hart;P. Stone;R. Mooney
在这项工作中，我们提出了使用人机对话，以提高语言理解的移动的机器人代理的方法。智能体将自然语言解析为潜在的语义，并使用机器人传感器来创建红色和沉重等感知概念的多模态模型。该代理可以用于显示导航路线，将对象交付给人们，并将对象从一个位置重新定位到另一个位置。我们使用对话澄清问题来理解命令和生成额外的解析训练数据。智能体采用机会主动学习来选择关于单词如何与对象相关的问题，从而提高其对感知概念的理解。我们在Amazon Mechanical Turk上对该代理进行了评估。在对对话中产生的数据进行培训后，该代理减少了询问的对话问题数量，同时获得了更高的可用性评级。此外，我们在机器人平台上演示了智能体，在那里它在完成现实世界任务的同时学习了新的感知概念。
In this work, we present methods for using human-robot dialog to improve language understanding for a mobile robot agent. The agent parses natural language to underlying semantic meanings and uses robotic sensors to create multi-modal models of perceptual concepts like red and heavy. The agent can be used for showing navigation routes, delivering objects to people, and relocating objects from one location to another. We use dialog clari_cation questions both to understand commands and to generate additional parsing training data. The agent employs opportunistic active learning to select questions about how words relate to objects, improving its understanding of perceptual concepts. We evaluated this agent on Amazon Mechanical Turk. After training on data induced from conversations, the agent reduced the number of dialog questions it asked while receiving higher usability ratings. Additionally, we demonstrated the agent on a robotic platform, where it learned new perceptual concepts on the y while completing a real-world task.