NRI: INT: COLLAB: Collaborative Task Planning and Learning through Language Communication in a Human-Robot Team

NRI：INT：COLLAB：人机团队中通过语言交流进行协作任务规划和学习

• 批准号: 1949634
• 负责人: Joyce Chai
• 金额: $ 76.67万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-07-15 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1949634&HistoricalAwards=false
• 关键词: NRI; INT; COLLAB; Collaborative; Task

When deployed in the field, robots will often encounter new situations or new tasks they don't have any knowledge or experience about. Even given sufficient knowledge, designing planners that can generate high quality plans and perform efficiently across various domains remains an open challenge. To address these issues, this project aims to empower robots to harness human expertise to acquire new knowledge and to engage humans in the loop of plan generation so that humans and robots can collectively arrive at a joint plan. The results will lead to principles and computational models for enabling effective human-robot teams that can adapt to new and changing environments and tasks, which will benefit many applications such as manufacturing, service, assistive technology, and search and rescue. This project will also provide new exciting training and education opportunities for students through research mentoring and curriculum development. This project investigates how humans and robots strive to mediate goals, world models, and plans to establish common ground for joint tasks. It will develop a computational framework that tightly links language and dialogue processing with the robot's underlying planning system to support collaborative task planning and learning in a human-robot team. It further will evaluate collaborative model acquisition and plan generation in terms of consistency of shared understanding, plan quality, and situational awareness. The research will transform planning in a human-robot team by integrating human expertise and knowledge in a collaborative process to improve planning and task performance. It will endow the robot with an ability to explain its internal states, goals and plans, and to continuously learn new states, actions, and plans through language communication with human partners. It will also advance language and dialogue research by providing a rich context for studying grounded semantics of language.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当部署在现场时，机器人经常会遇到他们没有任何知识或经验的新情况或新任务。即使有了足够的知识，设计能够生成高质量计划并在各个领域有效执行的规划者仍然是一个悬而未决的挑战。为了解决这些问题，该项目旨在使机器人能够利用人类的专业知识来获取新的知识，并让人类参与计划生成的循环，以便人类和机器人能够共同达成一项联合计划。研究结果将产生使有效的人-机器人团队能够适应新的和不断变化的环境和任务的原则和计算模型，这将使许多应用受益，如制造、服务、辅助技术和搜索和救援。该项目还将通过研究、指导和课程开发为学生提供新的令人兴奋的培训和教育机会。这个项目调查了人类和机器人如何努力调解目标、世界模型，并计划为联合任务建立共同点。它将开发一个计算框架，将语言和对话处理与机器人的底层规划系统紧密联系起来，以支持人类-机器人团队中的协作任务规划和学习。它还将从共同理解、计划质量和情景感知的一致性方面对协作模型获取和计划生成进行评估。这项研究将通过在协作过程中整合人类的专业知识和知识来改善规划和任务绩效，从而改变人类-机器人团队的规划。它将赋予机器人解释其内部状态、目标和计划的能力，并通过与人类合作伙伴的语言交流不断学习新的状态、动作和计划。它还将通过为研究扎根的语言语义提供丰富的背景来推动语言和对话研究。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Learning to Mediate Disparities Towards Pragmatic Communication

• DOI: 10.48550/arxiv.2203.13685
• 发表时间: 2022-03
• 期刊: ArXiv
• 作者: Yuwei Bao;Sayan Ghosh;J. Chai

Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring

使用统一变压器和自我监控从语言指令中学习分层任务

• DOI: 10.18653/v1/2021.findings-acl.368
• 发表时间: 2021
• 期刊: Findings.
• 作者: Zhang, Yichi;Chai, Joyce
• 通讯作者: Chai, Joyce

Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models

走向大语言模型中情境心理理论的整体景观

• 发表时间: 2023
• 期刊: Findings of Empirical Methods in Natural Language Processing
• 作者: Ma, Ziqiao;Sansom, Jacob;Peng, Run;Chai, Joyce
• 通讯作者: Chai, Joyce

Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue

• DOI: 10.24963/ijcai.2023/330
• 发表时间: 2023-05
• 作者: Cristian-Paul Bara;Ziqiao Ma;Yingzhuo Yu;J. Shah;J. Chai

DANLI: Deliberative Agent for Following Natural Language Instructions

• DOI: 10.48550/arxiv.2210.12485
• 发表时间: 2022-10
• 期刊: Environ. Model. Softw.
• 作者: Yichi Zhang;Jianing Yang;Jiayi Pan;Shane Storks;N. Devraj;Ziqiao Ma;Keunwoo Peter Yu;Yuwei Bao;J. Chai