NRI: Collaborative Research: Learning Adaptive Representations for Robust Mobile Robot Navigation from Multi-Modal Interactions

NRI：协作研究：从多模态交互中学习鲁棒移动机器人导航的自适应表示

• 批准号: 1637813
• 负责人: Thomas Howard
• 金额: $ 28.94万
• 依托单位: University of Rochester
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1637813&HistoricalAwards=false
• 关键词: NRI; Collaborative; Research; Learning; Adaptive

Most existing autonomous systems reason over flat, task-dependent models of the world that do not scale to large, complex environments. This lack of scalability and generalizability is a significant barrier to the widespread adoption of robots for common tasks. This research will advance the state-of-the-art in robot perception, natural language understanding, and learning to develop new models and algorithms that significantly improve the scalability and efficiency of mapping and motion planning in large, complex environments. These contributions will impact the next generation of autonomous systems that interact with humans in many domains, including manufacturing, healthcare, and exploration. Outcomes will include the release of open source software and data, workshops, K-12 STEM outreach efforts, and undergraduate and graduate education in the unique, multidisciplinary fields of perception, natural language understanding, and motion planning. As robots perform a wider variety of tasks within increasingly complex environments, their ability to learn and reason over expressive models of their environment becomes critical. The goal of this research is to develop models and algorithms for learning adaptive, hierarchical environment representations that afford efficient planning for mobility tasks. These representations will take the form of probabilistic models that capture the rich spatial-semantic properties of the robot's environment and are factorable to enable scalable inference. This research will develop algorithms that learn and adapt these representations by fusing knowledge conveyed through human-provided natural language utterances with information extracted from the robot's multimodal sensor streams. This research will develop algorithms that then reason over the complexity of these models in the context of the inferred task, thereby identifying simplifications that enable more efficient robot motion planning.

大多数现有的自主系统是基于扁平的、依赖于任务的世界模型进行推理的，这些模型不能扩展到大型、复杂的环境。这种缺乏可伸缩性和通用性是广泛采用机器人执行常见任务的重大障碍。这项研究将推动机器人感知、自然语言理解以及学习开发新模型和算法的最新进展，这些模型和算法可以显著提高大型复杂环境中地图绘制和运动规划的可扩展性和效率。这些贡献将影响下一代自主系统，这些系统在许多领域与人类互动，包括制造、医疗保健和探索。成果将包括发布开源软件和数据、研讨会、K-12 STEM推广工作，以及在感知、自然语言理解和动作规划等独特的多学科领域的本科生和研究生教育。随着机器人在日益复杂的环境中执行更多种类的任务，它们学习和推理环境表达模型的能力变得至关重要。这项研究的目标是开发模型和算法来学习自适应的、分层的环境表征，从而为移动任务提供有效的规划。这些表示将采用概率模型的形式，该模型捕捉机器人环境的丰富空间语义属性，并可因式分解以实现可扩展的推理。这项研究将开发算法，通过融合通过人类提供的自然语言话语传达的知识和从机器人的多模式传感器流中提取的信息来学习和适应这些表示。这项研究将开发算法，然后在推断的任务背景下对这些模型的复杂性进行推理，从而确定能够更有效地进行机器人运动规划的简化。

项目成果

Language-Guided Adaptive Perception for Efficient Grounded Communication with Robotic Manipulators in Cluttered Environments

语言引导的自适应感知，可在杂乱的环境中与机器人操纵器进行有效的接地通信

• 发表时间: 2018
• 期刊: 19th Annual SIGdial Meeting on Discourse and Dialogue
• 作者: Patki, Siddharth;Howard, Thomas M.
• 通讯作者: Howard, Thomas M.

Learning Models for Predictive Adaptation in State Lattices

状态格中预测适应的学习模型

• 发表时间: 2018
• 期刊: 11th International Conference on Field and Service Robotics
• 作者: Napoli, Michael E.;Biggie, Harel;Howard, Thomas M.
• 通讯作者: Howard, Thomas M.

A Multiview Approach to Learning Articulated Motion Models

• DOI: 10.1007/978-3-030-28619-4_30
• 发表时间: 2017
• 期刊: Mitochondrion
• 影响因子: 4.4
• 作者: Andrea F. Daniele;T. Howard;Matthew R. Walter

Language-guided Semantic Mapping and Mobile Manipulation in Partially Observable Environments

部分可观察环境中的语言引导语义映射和移动操作

• 发表时间: 2020
• 期刊: Proceedings of Machine Learning Research
• 作者: Patki, Siddharth;Fahnestock, Ethan;Howard, Thomas M.;Walter, Matthew R.
• 通讯作者: Walter, Matthew R.

Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions

推断紧凑表示以有效理解机器人指令的自然语言

• 发表时间: 2019
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Patki, Siddharth;Daniele, Andrea F;Walter, Matthew R;Howard, Thomas M
• 通讯作者: Howard, Thomas M