CRII: RI: Joint Models of Language and Context for Robotic Language Acquisition

随着机器人变得更小，更便宜，更有能力，它们能够执行越来越多的任务，从而导致汽车安全和制造等领域的革命性改进。然而，它们的可扩展性使得它们难以部署在以人为中心的环境中，例如家庭和学校，在这些环境中，它们的任务和环境不断变化。与此同时，学习理解物理世界的语言是机器人和自然语言处理中一个不断发展的研究领域。核心问题是，在机器人所处的嘈杂、感知的世界中，单词的含义是如何建立起来的。该项目探讨了机器人如何从自然语言中了解世界，以便从人类那里自然直观地接受指令并了解其环境。遵循方向的能力减少了机器人在辅助技术、教育和护理等领域的采用障碍，在这些领域，与非专业人士的互动至关重要。这种机器人有可能最终提高老年人等人群的自主性和独立性;例如，可以向用户学习的机械臂？的解释如何处理食物或打开新的容器将直接影响独立的人与灵巧的关注，如先进的关节炎。这是一个探索性的调查，语言和感知模型如何在交互过程中扩展，使机器人能够理解有关意想不到的领域的新语言。特别是，重点是开发新的学习方法，正确地诱导语言和感知的联合模型，建立数据驱动的语言模型，随着时间的推移添加新的语义表示。这项工作结合了语义解析器学习，它提供了一个分布在可能的解释语言，与感知表征的基础世界。当遇到新的单词和新的感知数据时，新的概念被动态地添加，并且可以通过最大化语言和视觉组件的预期可能性来训练语义上有意义的模型。这种集成方法允许有效的模型更新，而无需明确标记单词或感知。这种方法将与通过结合主动学习来提高学习效率的实验相结合，利用机器人对世界上的物体提出问题的能力。

As robots become smaller, less expensive, and more capable, they are able to perform an increasing variety of tasks, leading to revolutionary improvements in domains such as automobile safety and manufacturing. However, their inflexibility makes them hard to deploy in human-centric environments such as homes and schools, where their tasks and environments are constantly changing. Meanwhile, learning to understand language about the physical world is a growing research area in both robotics and natural language processing. The core problem is how the meanings of words are grounded in the noisy, perceptual world in which a robot operates. This project explores how robots can learn about the world from natural language in order to take instructions and learn about their environment naturally and intuitively from people. The ability to follow directions reduces the adoption barrier for robots in domains such as assistive technology, education, and caretaking, where interactions with non-specialists are crucial. Such robots have the potential to ultimately improve autonomy and independence for populations such as aging-in-place elders; for example, a manipulator arm that can learn from a user?s explanation how to handle food or open novel containers would directly affect the independence of persons with dexterity concerns such as advanced arthritis. This is an exploratory investigation of how linguistic and perceptual models can be expanded during interaction, allowing robots to understand novel language about unanticipated domains. In particular, the focus is on developing new learning approaches that correctly induce joint models of language and perception, building data-driven language models that add new semantic representations over time. The work combines semantic parser learning, which provides a distribution over possible interpretations of language, with perceptual representations of the underlying world. New concepts are added on the fly as new words and new perceptual data are encountered, and a semantically meaningful model can be trained by maximizing the expected likelihood of language and visual components. This integrated approach allows for effective model updates with no explicit labeling of words or percepts. This approach will be combined with experiments on improving learning efficiency by incorporating active learning, leveraging a robot's ability to ask questions about objects in the world.