NRI: Collaborative Research: Learning Adaptive Representations for Robust Mobile Robot Navigation from Multi-Modal Interactions

大多数现有的自主系统是基于扁平的、依赖于任务的世界模型进行推理的，这些模型不能扩展到大型、复杂的环境。这种缺乏可伸缩性和通用性是广泛采用机器人执行常见任务的重大障碍。这项研究将推动机器人感知、自然语言理解以及学习开发新模型和算法的最新进展，这些模型和算法可以显著提高大型复杂环境中地图绘制和运动规划的可扩展性和效率。这些贡献将影响下一代自主系统，这些系统在许多领域与人类互动，包括制造、医疗保健和探索。成果将包括发布开源软件和数据、研讨会、K-12 STEM推广工作，以及在感知、自然语言理解和动作规划等独特的多学科领域的本科生和研究生教育。随着机器人在日益复杂的环境中执行更多种类的任务，它们学习和推理环境表达模型的能力变得至关重要。这项研究的目标是开发模型和算法来学习自适应的、分层的环境表征，从而为移动任务提供有效的规划。这些表示将采用概率模型的形式，该模型捕捉机器人环境的丰富空间语义属性，并可因式分解以实现可扩展的推理。这项研究将开发算法，通过融合通过人类提供的自然语言话语传达的知识和从机器人的多模式传感器流中提取的信息来学习和适应这些表示。这项研究将开发算法，然后在推断的任务背景下对这些模型的复杂性进行推理，从而确定能够更有效地进行机器人运动规划的简化。

Most existing autonomous systems reason over flat, task-dependent models of the world that do not scale to large, complex environments. This lack of scalability and generalizability is a significant barrier to the widespread adoption of robots for common tasks. This research will advance the state-of-the-art in robot perception, natural language understanding, and learning to develop new models and algorithms that significantly improve the scalability and efficiency of mapping and motion planning in large, complex environments. These contributions will impact the next generation of autonomous systems that interact with humans in many domains, including manufacturing, healthcare, and exploration. Outcomes will include the release of open source software and data, workshops, K-12 STEM outreach efforts, and undergraduate and graduate education in the unique, multidisciplinary fields of perception, natural language understanding, and motion planning. As robots perform a wider variety of tasks within increasingly complex environments, their ability to learn and reason over expressive models of their environment becomes critical. The goal of this research is to develop models and algorithms for learning adaptive, hierarchical environment representations that afford efficient planning for mobility tasks. These representations will take the form of probabilistic models that capture the rich spatial-semantic properties of the robot's environment and are factorable to enable scalable inference. This research will develop algorithms that learn and adapt these representations by fusing knowledge conveyed through human-provided natural language utterances with information extracted from the robot's multimodal sensor streams. This research will develop algorithms that then reason over the complexity of these models in the context of the inferred task, thereby identifying simplifications that enable more efficient robot motion planning.