NRI: Collaborative Research: Learning Adaptive Representations for Robust Mobile Robot Navigation from Multi-Modal Interactions

NRI：协作研究：从多模态交互中学习鲁棒移动机器人导航的自适应表示

• 批准号: 1638072
• 负责人: Matthew Walter
• 金额: $ 33.27万
• 依托单位: Toyota Technological Institute at Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-10-01 至 2021-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1638072&HistoricalAwards=false
• 关键词: NRI; Collaborative; Research; Learning; Adaptive

Most existing autonomous systems reason over flat, task-dependent models of the world that do not scale to large, complex environments. This lack of scalability and generalizability is a significant barrier to the widespread adoption of robots for common tasks. This research will advance the state-of-the-art in robot perception, natural language understanding, and learning to develop new models and algorithms that significantly improve the scalability and efficiency of mapping and motion planning in large, complex environments. These contributions will impact the next generation of autonomous systems that interact with humans in many domains, including manufacturing, healthcare, and exploration. Outcomes will include the release of open source software and data, workshops, K-12 STEM outreach efforts, and undergraduate and graduate education in the unique, multidisciplinary fields of perception, natural language understanding, and motion planning.As robots perform a wider variety of tasks within increasingly complex environments, their ability to learn and reason over expressive models of their environment becomes critical. The goal of this research is to develop models and algorithms for learning adaptive, hierarchical environment representations that afford efficient planning for mobility tasks. These representations will take the form of probabilistic models that capture the rich spatial-semantic properties of the robot's environment and are factorable to enable scalable inference. This research will develop algorithms that learn and adapt these representations by fusing knowledge conveyed through human-provided natural language utterances with information extracted from the robot's multimodal sensor streams. This research will develop algorithms that then reason over the complexity of these models in the context of the inferred task, thereby identifying simplifications that enable more efficient robot motion planning.

大多数现有的自主系统都基于扁平的、依赖于任务的世界模型进行推理，这些模型无法扩展到大型、复杂的环境。缺乏可扩展性和通用性是广泛采用机器人执行常见任务的重大障碍。这项研究将推进机器人感知、自然语言理解以及学习开发新模型和算法方面的最新技术，从而显着提高大型复杂环境中绘图和运动规划的可扩展性和效率。这些贡献将影响下一代在许多领域与人类互动的自主系统，包括制造、医疗保健和探索。 成果将包括开源软件和数据的发布、研讨会、K-12 STEM 推广工作以及独特的多学科感知、自然语言理解和运动规划领域的本科生和研究生教育。随着机器人在日益复杂的环境中执行更广泛的任务，它们学习和推理环境表达模型的能力变得至关重要。这项研究的目标是开发用于学习自适应、分层环境表示的模型和算法，为移动任务提供有效的规划。这些表示将采用概率模型的形式，捕获机器人环境的丰富空间语义属性，并可分解以实现可扩展的推理。这项研究将开发算法，通过将人类提供的自然语言表达所传达的知识与从机器人多模态传感器流中提取的信息融合起来，学习和适应这些表示。这项研究将开发算法，然后在推断任务的背景下对这些模型的复杂性进行推理，从而确定能够实现更高效的机器人运动规划的简化方法。

项目成果

Integrated Benchmarking and Design for Reproducible and Accessible Evaluation of Robotic Agents

机器人代理可重复且可访问评估的集成基准测试和设计

• DOI: 10.1109/iros45743.2020.9341677
• 发表时间: 2020
• 期刊: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS
• 作者: Tani, Jacopo;Daniele, Andrea F.;Bernasconi, Gianmarco;Camus, Amaury;Petrov, Aleksandar;Courchesne, Anthony;Mehta, Bhairav;Suri, Rohit;Zaluska, Tomasz;Walter, Matthew R.
• 通讯作者: Walter, Matthew R.

Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning

• DOI: 10.1109/icra.2019.8793537
• 发表时间: 2018-01
• 期刊: 2019 International Conference on Robotics and Automation (ICRA)
• 作者: Charles B. Schaff;David Yunis;Ayan Chakrabarti;Matthew R. Walter

Multimodal estimation and communication of latent semantic knowledge for robust execution of robot instructions

• DOI: 10.1177/0278364920917755
• 发表时间: 2020-06-05
• 期刊: INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH
• 影响因子: 9.2
• 作者: Arkin, Jacob;Park, Daehyung;Paul, Rohan
• 通讯作者: Paul, Rohan

Language-guided Semantic Mapping and Mobile Manipulation in Partially Observable Environments

部分可观察环境中的语言引导语义映射和移动操作

• 发表时间: 2020
• 期刊: Proceedings of Machine Learning Research
• 作者: Patki, Siddharth;Fahnestock, Ethan;Howard, Thomas M.;Walter, Matthew R.
• 通讯作者: Walter, Matthew R.

Inferring Compact Representations for Efficient Natural Language Understanding of Robot Instructions

推断紧凑表示以有效理解机器人指令的自然语言

• 发表时间: 2019
• 期刊: IEEE International Conference on Robotics and Automation
• 作者: Patki, Siddharth;Daniele, Andrea F;Walter, Matthew R;Howard, Thomas M
• 通讯作者: Howard, Thomas M