CRII: RI: Joint Models of Language and Context for Robotic Language Acquisition

随着机器人变得更小、更便宜、更有能力，它们能够执行越来越多的任务，导致汽车安全和制造业等领域的革命性改进。然而，它们的不灵活性使得它们难以部署在以人为中心的环境中，例如家庭和学校，在这些环境中，它们的任务和环境不断变化。与此同时，学习理解物理世界的语言是机器人和自然语言处理领域一个不断发展的研究领域。核心问题是，单词的含义如何在机器人操作的嘈杂、感知的世界中建立基础。这个项目探索了机器人如何从自然语言中了解世界，以便从人类那里自然而直观地接受指令和了解环境。在辅助技术、教育和看护等领域，与非专业人员的互动至关重要，听从指令的能力减少了机器人的采用障碍。这类机器人有可能最终提高老年人等人群的自主性和独立性；比如，一个可以向使用者学习的机械臂？对于如何处理食物或打开新容器的解释，将直接影响到那些有灵巧性问题的人的独立性，比如晚期关节炎。这是一项关于语言和感知模型如何在交互过程中扩展的探索性研究，使机器人能够理解意想不到领域的新语言。特别是，重点是开发新的学习方法，正确地归纳语言和感知的联合模型，建立数据驱动的语言模型，随着时间的推移增加新的语义表示。这项工作结合了语义解析器学习，它提供了语言可能解释的分布，以及底层世界的感知表征。当遇到新的单词和新的感知数据时，可以动态地添加新的概念，并且可以通过最大化语言和视觉组件的预期可能性来训练语义上有意义的模型。这种集成方法允许有效的模型更新，而不需要明确标记单词或感知器。这种方法将与通过主动学习来提高学习效率的实验相结合，利用机器人对世界上的物体提出问题的能力。

As robots become smaller, less expensive, and more capable, they are able to perform an increasing variety of tasks, leading to revolutionary improvements in domains such as automobile safety and manufacturing. However, their inflexibility makes them hard to deploy in human-centric environments such as homes and schools, where their tasks and environments are constantly changing. Meanwhile, learning to understand language about the physical world is a growing research area in both robotics and natural language processing. The core problem is how the meanings of words are grounded in the noisy, perceptual world in which a robot operates. This project explores how robots can learn about the world from natural language in order to take instructions and learn about their environment naturally and intuitively from people. The ability to follow directions reduces the adoption barrier for robots in domains such as assistive technology, education, and caretaking, where interactions with non-specialists are crucial. Such robots have the potential to ultimately improve autonomy and independence for populations such as aging-in-place elders; for example, a manipulator arm that can learn from a user?s explanation how to handle food or open novel containers would directly affect the independence of persons with dexterity concerns such as advanced arthritis. This is an exploratory investigation of how linguistic and perceptual models can be expanded during interaction, allowing robots to understand novel language about unanticipated domains. In particular, the focus is on developing new learning approaches that correctly induce joint models of language and perception, building data-driven language models that add new semantic representations over time. The work combines semantic parser learning, which provides a distribution over possible interpretations of language, with perceptual representations of the underlying world. New concepts are added on the fly as new words and new perceptual data are encountered, and a semantically meaningful model can be trained by maximizing the expected likelihood of language and visual components. This integrated approach allows for effective model updates with no explicit labeling of words or percepts. This approach will be combined with experiments on improving learning efficiency by incorporating active learning, leveraging a robot's ability to ask questions about objects in the world.