CRII: RI: Joint Models of Language and Context for Robotic Language Acquisition

随着机器人变得更小、更便宜、更有能力，它们能够执行越来越多的任务，导致汽车安全和制造等领域的革命性改进。然而，它们缺乏灵活性，使得它们很难部署在以人为中心的环境中，如家庭和学校，在这些环境中，它们的任务和环境不断变化。与此同时，学习理解关于物理世界的语言是机器人学和自然语言处理领域中一个日益增长的研究领域。核心问题是，单词的意义如何植根于机器人操作的嘈杂、感知的世界中。这个项目探索了机器人如何从自然语言中学习世界，以便接受指令，并自然、直观地从人类那里了解他们的环境。遵循指令的能力减少了机器人在辅助技术、教育和护理等领域的采用障碍，在这些领域，与非专家的互动至关重要。这类机器人有可能最终提高老年人等群体的自主性和独立性；例如，可以向用户学习的机械手？S解释说，如何处理食物或打开新颖的容器将直接影响患有晚期关节炎等灵巧性问题的人的独立性。这是一项关于语言和感知模型如何在交互过程中扩展的探索性调查，使机器人能够理解关于未预料到的领域的新语言。特别是，重点是开发新的学习方法，正确地归纳语言和感知的联合模型，建立数据驱动的语言模型，随着时间的推移添加新的语义表示。这项工作结合了语义解析器学习，它提供了对语言的可能解释的分布，以及对底层世界的感知表示。当遇到新词和新的感知数据时，新概念在运行中被添加，并且可以通过最大化语言和视觉组件的预期似然来训练语义有意义的模型。这种集成的方法允许有效的模型更新，而不需要对单词或感知进行明确的标记。这种方法将与通过融入主动学习来提高学习效率的实验相结合，利用机器人询问世界上物体的能力。

As robots become smaller, less expensive, and more capable, they are able to perform an increasing variety of tasks, leading to revolutionary improvements in domains such as automobile safety and manufacturing. However, their inflexibility makes them hard to deploy in human-centric environments such as homes and schools, where their tasks and environments are constantly changing. Meanwhile, learning to understand language about the physical world is a growing research area in both robotics and natural language processing. The core problem is how the meanings of words are grounded in the noisy, perceptual world in which a robot operates. This project explores how robots can learn about the world from natural language in order to take instructions and learn about their environment naturally and intuitively from people. The ability to follow directions reduces the adoption barrier for robots in domains such as assistive technology, education, and caretaking, where interactions with non-specialists are crucial. Such robots have the potential to ultimately improve autonomy and independence for populations such as aging-in-place elders; for example, a manipulator arm that can learn from a user?s explanation how to handle food or open novel containers would directly affect the independence of persons with dexterity concerns such as advanced arthritis. This is an exploratory investigation of how linguistic and perceptual models can be expanded during interaction, allowing robots to understand novel language about unanticipated domains. In particular, the focus is on developing new learning approaches that correctly induce joint models of language and perception, building data-driven language models that add new semantic representations over time. The work combines semantic parser learning, which provides a distribution over possible interpretations of language, with perceptual representations of the underlying world. New concepts are added on the fly as new words and new perceptual data are encountered, and a semantically meaningful model can be trained by maximizing the expected likelihood of language and visual components. This integrated approach allows for effective model updates with no explicit labeling of words or percepts. This approach will be combined with experiments on improving learning efficiency by incorporating active learning, leveraging a robot's ability to ask questions about objects in the world.