NRI: Robots that Learn to Communicate through Natural Human Dialog

机器人的能力越来越强，并即将成为一种无处不在的技术。为了让机器人真正有用，人们必须能够用日常人类语言有效地交流他们的需求。尽管有越来越多的人与机器人交互的自然语言处理研究，但它通常需要由工程专家提供某种形式的显式监督，并涉及非自然的、艰苦的培训，以获得健壮性和覆盖率。该项目涉及开发人-机器人对话系统，该系统学习通过自然对话与用户交流，从重复的正常用户交互中学习，变得更健壮和有能力。该项目支持学生在自然语言处理、人机交互和机器学习领域的教育，这些领域对受过教育的人员有很大的需求。它与该大学的新生研究倡议相结合，该倡议让本科生在第一年参与研究。为了开发通过正常用户交互学习提高沟通技能的人-机器人对话系统，该项目整合并调整了目前三个不同技术领域的学习技术：语义分析、口语对话管理和感知语言基础。该项目采用并集成了使用组合范畴语法(CCG)的语义解析器学习、使用部分可观察的马尔可夫决策过程(POMDP)的对话管理以及使用视觉和触觉传感器的多模式语言基础的技术，以开发用于与机器人通信的对话系统，该对话系统包括德克萨斯大学奥斯汀分校正在开发的建筑广泛智能(BWI)系统。这项研究将PI在语义解析和语言基础方面的专业知识与共同PI在机器人学和强化学习方面的专业知识结合在一起，形成了一个独特的跨学科团队，为人-机器人交互开发新颖而有效的系统。该项目包括使用受控实验对一系列任务进行严格的评估，使用众包用户的在线模拟，以及用户与移动机器人平台的自然交互，移动机器人平台由为BWI系统开发的轮式Segway底座和Kinova机器人手臂组成。

Robots are increasingly capable and are on the threshold of becoming a ubiquitous technology. For robots to be truly useful, people must be able to effectively communicate their needs in everyday human language. Although there is a growing body of research on natural-language processing for human-robot interaction, it typically requires some form of explicit supervision provided by an engineering expert and involves unnatural, laborious training to obtain robustness and coverage. This project involves the development of human-robot dialog systems that learn to communicate with users through natural dialog, learning from repeated normal user interactions to become more robust and capable. The project supports the education of students in the areas of natural-language processing, human-robot interaction, and machine learning, where there is significant demand for educated personnel. It is integrated with the university's Freshman Research Initiative, which gets undergraduate students involved in research in their first year.In order to develop human-robot dialog systems that learn to improve their communication skills through normal user interactions, the project integrates and adapts learning techniques from three currently disparate technical areas: semantic parsing, spoken dialog management, and perceptual language grounding. The project adapts and integrates techniques for semantic-parser learning using combinatory categorial grammar (CCG), dialog management using Partially Observable Markov Decision Processes (POMDPs), and multi-modal language grounding using both visual and haptic sensors, in order to develop a dialog system for communicating with robots that comprise the Building Wide Intelligence (BWI) system being developed at the University of Texas at Austin. The research integrates the PI's expertise in semantic parsing and language grounding with the co-PI's expertise in robotics and reinforcement learning, forming a unique interdisciplinary team for developing novel and effective systems for human-robot interaction. The project includes rigorous evaluations using controlled experiments on a range of tasks using both on-line simulations with crowdsourced users, and natural user interaction with a mobile robot platform consisting of a wheeled Segway base and a Kinova robot arm being developed for the BWI system.