EAGER: Robots that Learn to Communicate with Humans Tthrough Natural Dialog

EAGER：通过自然对话学习与人类交流的机器人

• 批准号: 1548567
• 负责人: Raymond Mooney
• 金额: $ 15万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1548567&HistoricalAwards=false
• 关键词: EAGER; Robots; Learn; Communicate; Humans

This EArly Grant for Exploratory Research explores the possibility of developing more user-friendly and capable robots that learn to understand commands in natural human language. The experimental system developed aims to engage users in natural conversation, clarifying linguistic instructions that cannot be understood, and learning from this interaction to more robustly interpret future commands. This fundamentally new approach is hypothesized to overcome limitations of more-costly previous approaches that require either direct programming or detailed annotation of per-assembled linguistic data, and still frequently fail to cover issues that arise in real user interactions. The resulting exploratory prototype is evaluated on real interactions with human users, experimentally testing its ability to improve its accuracy and flexibility at interpreting human instructions over time, through normal everyday use. This novel approach aims to improve human interaction with intelligent multi-robot systems that aid the residents and visitors of a large, multi-use building. This fundamental research also supports computer-science education in the growing areas of natural-language processing, human-robot interaction, and machine learning, where there is significant national demand for knowledgeable personnel.The technical approach explored is a novel integration of learning techniques from three currently disparate areas: semantic parsing, spoken dialog management, and perceptual language grounding. Semantic parsing is the task of mapping natural language to a formal computer-interpretable language using compositional semantics based on syntactic linguistic structure. Dialog management concerns controlling multi-turn natural language interaction to aid comprehension and task completion. Perceptual grounding concerns associating words and phrases in language to objects, properties and relations in the world as perceived by the robot's sensors. Although there has been recent significant progress in each of these individual areas, no one has previously explored integrating them to support learning for human-robot communication through natural dialog. This exploratory research adapts and integrates techniques for semantic-parser learning using combinatory categorial grammar, dialog management using Partially Observable Markov Decision Processes, and multi-modal language grounding using both visual and haptic sensors, in order to develop a novel dialog system for communicating with robots that comprise the innovative Building Wide Intelligence system being developed at the University of Texas at Austin. The exploratory methods are evaluated using controlled experiments on a range of tasks using both on-line simulations and crowdsourced users, and natural user interaction with a mobile robot platform consisting of a wheeled Segway base and a Kinova robot arm.

EArly的探索性研究资助探索了开发更用户友好和有能力的机器人的可能性，这些机器人可以学习理解自然人类语言中的命令。开发的实验系统旨在让用户参与自然对话，澄清无法理解的语言指令，并从这种交互中学习，以更强大地解释未来的命令。假设这种全新的方法可以克服成本更高的先前方法的局限性，这些方法需要直接编程或详细注释预组装的语言数据，并且仍然经常无法覆盖真实的用户交互中出现的问题。由此产生的探索性原型评估与人类用户的真实的交互，实验测试其能力，以提高其准确性和灵活性，随着时间的推移，通过正常的日常使用解释人类的指令。这种新颖的方法旨在改善人类与智能多机器人系统的互动，帮助大型多用途建筑的居民和游客。这项基础研究也支持计算机科学教育在自然语言处理，人机交互和机器学习，有显着的国家需求的知识渊博的人才。探索的技术方法是一种新的集成学习技术从三个目前不同的领域：语义解析，口语对话管理，感知语言基础。语义分析是基于句法语言结构，使用组合语义将自然语言映射为正式的计算机可解释语言的任务。对话管理涉及控制多轮自然语言交互以帮助理解和任务完成。感知基础涉及将语言中的单词和短语与机器人传感器感知到的世界中的对象，属性和关系相关联。虽然最近在这些领域都取得了重大进展，但以前没有人探索过将它们集成到支持通过自然对话进行人机交流的学习中。这项探索性的研究适应和整合语义分析器学习使用组合类别语法，对话管理使用部分可观察马尔可夫决策过程，和多模态语言接地使用视觉和触觉传感器的技术，以开发一种新的对话系统与机器人进行通信，包括创新的建筑物广泛的智能系统正在开发的得克萨斯大学奥斯汀分校。探索性的方法进行评估，使用控制实验的一系列任务，使用在线模拟和众包用户，和自然的用户交互与移动的机器人平台组成的轮式赛格威基地和Kinova机器人手臂。

项目成果

Multi-Modal Word Synset Induction

多模态词同义词集归纳

• 发表时间: 2017
• 期刊: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17
• 作者: Tomason, J.;Mooney, R.J.
• 通讯作者: Mooney, R.J.