NRI: Collaborative Research: Learning Adaptive Representations for Robust Mobile Robot Navigation from Multi-Modal Interactions

大多数现有的自主系统都是基于平面的、依赖于任务的世界模型，这些模型不能扩展到大型复杂的环境中。这种可扩展性和通用性的缺乏是广泛采用机器人执行常见任务的一个重大障碍。这项研究将推进机器人感知，自然语言理解和学习的最新技术，以开发新的模型和算法，显着提高大型复杂环境中映射和运动规划的可扩展性和效率。这些贡献将影响下一代自主系统，这些系统在许多领域与人类互动，包括制造业，医疗保健和探索。成果将包括开源软件和数据的发布，研讨会，K-12 STEM外展工作，以及感知，自然语言理解和运动规划等独特的多学科领域的本科和研究生教育。随着机器人在日益复杂的环境中执行各种各样的任务，它们对环境的表达模型进行学习和推理的能力变得至关重要。本研究的目标是开发模型和算法，学习自适应，分层的环境表示，提供有效的规划移动任务。这些表示将采用概率模型的形式，这些概率模型捕获机器人环境的丰富空间语义属性，并且是可分解的，以实现可扩展的推理。这项研究将开发算法，通过将人类提供的自然语言话语传达的知识与从机器人的多模态传感器流中提取的信息融合，来学习和适应这些表示。这项研究将开发算法，然后在推断任务的背景下对这些模型的复杂性进行推理，从而确定简化，使机器人运动规划更有效。

Most existing autonomous systems reason over flat, task-dependent models of the world that do not scale to large, complex environments. This lack of scalability and generalizability is a significant barrier to the widespread adoption of robots for common tasks. This research will advance the state-of-the-art in robot perception, natural language understanding, and learning to develop new models and algorithms that significantly improve the scalability and efficiency of mapping and motion planning in large, complex environments. These contributions will impact the next generation of autonomous systems that interact with humans in many domains, including manufacturing, healthcare, and exploration. Outcomes will include the release of open source software and data, workshops, K-12 STEM outreach efforts, and undergraduate and graduate education in the unique, multidisciplinary fields of perception, natural language understanding, and motion planning. As robots perform a wider variety of tasks within increasingly complex environments, their ability to learn and reason over expressive models of their environment becomes critical. The goal of this research is to develop models and algorithms for learning adaptive, hierarchical environment representations that afford efficient planning for mobility tasks. These representations will take the form of probabilistic models that capture the rich spatial-semantic properties of the robot's environment and are factorable to enable scalable inference. This research will develop algorithms that learn and adapt these representations by fusing knowledge conveyed through human-provided natural language utterances with information extracted from the robot's multimodal sensor streams. This research will develop algorithms that then reason over the complexity of these models in the context of the inferred task, thereby identifying simplifications that enable more efficient robot motion planning.