NRI: Robots that Learn to Communicate through Natural Human Dialog

机器人的能力越来越强，即将成为一种无处不在的技术。为了让机器人真正有用，人们必须能够用日常人类语言有效地传达他们的需求。虽然越来越多的研究针对人机交互的自然语言处理，但它通常需要工程专家提供某种形式的明确监督，并且涉及非自然的，费力的训练以获得鲁棒性和覆盖率。该项目涉及人机对话系统的开发，该系统学习通过自然对话与用户进行交流，从重复的正常用户交互中学习，以变得更加强大和强大。该项目支持自然语言处理，人机交互和机器学习领域的学生教育，这些领域对受过教育的人员有很大的需求。 它与大学的新生研究计划相结合，该计划让本科生在第一年参与研究。为了开发通过正常的用户交互来学习提高沟通技能的人机对话系统，该项目整合并调整了目前三个不同技术领域的学习技术：语义解析，口语对话管理和感知语言基础。 该项目适应和集成的语义分析器学习使用组合分类语法（CCG），对话管理使用部分可观察马尔可夫决策过程（POMDPs），和多模态语言接地使用视觉和触觉传感器的技术，以开发一个对话系统，用于与机器人进行通信，包括建筑物广泛的智能（BWI）系统正在开发的得克萨斯大学奥斯汀分校。该研究将PI在语义解析和语言基础方面的专业知识与co-PI在机器人和强化学习方面的专业知识相结合，形成了一个独特的跨学科团队，用于开发新颖有效的人机交互系统。该项目包括严格的评估，使用控制实验的一系列任务，使用在线模拟与众包用户，和自然的用户互动的移动的机器人平台组成的轮式赛格威基地和Kinova机器人手臂正在开发的BWI系统。

Robots are increasingly capable and are on the threshold of becoming a ubiquitous technology. For robots to be truly useful, people must be able to effectively communicate their needs in everyday human language. Although there is a growing body of research on natural-language processing for human-robot interaction, it typically requires some form of explicit supervision provided by an engineering expert and involves unnatural, laborious training to obtain robustness and coverage. This project involves the development of human-robot dialog systems that learn to communicate with users through natural dialog, learning from repeated normal user interactions to become more robust and capable. The project supports the education of students in the areas of natural-language processing, human-robot interaction, and machine learning, where there is significant demand for educated personnel. It is integrated with the university's Freshman Research Initiative, which gets undergraduate students involved in research in their first year.In order to develop human-robot dialog systems that learn to improve their communication skills through normal user interactions, the project integrates and adapts learning techniques from three currently disparate technical areas: semantic parsing, spoken dialog management, and perceptual language grounding. The project adapts and integrates techniques for semantic-parser learning using combinatory categorial grammar (CCG), dialog management using Partially Observable Markov Decision Processes (POMDPs), and multi-modal language grounding using both visual and haptic sensors, in order to develop a dialog system for communicating with robots that comprise the Building Wide Intelligence (BWI) system being developed at the University of Texas at Austin. The research integrates the PI's expertise in semantic parsing and language grounding with the co-PI's expertise in robotics and reinforcement learning, forming a unique interdisciplinary team for developing novel and effective systems for human-robot interaction. The project includes rigorous evaluations using controlled experiments on a range of tasks using both on-line simulations with crowdsourced users, and natural user interaction with a mobile robot platform consisting of a wheeled Segway base and a Kinova robot arm being developed for the BWI system.