NRI: FND: Semi-Supervised Deep Learning for Domain Adaptation in Robotic Language Acquisition

该项目将使机器人能够从语言和其他人类模式中学习与人类队友一起执行任务，然后在不同的平台和任务中转移所学的知识。这最终将使人-机器人团队在人们使用不同语言和指令来完成复杂任务的领域成为可能。随着机器人的能力越来越强，无处不在，它们越来越多地进入工作场所和家庭等复杂的、以人为中心的环境。能够在辅助技术、老年人护理和教育等人类专家捉襟见肘的领域部署有用的机器人，有可能对人类的生活质量产生深远的影响。要实现这一目标，就需要开发能够从自然交互中学习最终用户目标和环境的机器人。这项工作的目的是让非专业人员更容易使用机器人。为了验证成功并让更广泛的社区参与进来，将从与教育和社区参与密切相关的社区创客空间中抽取任务并进行测试。该奖项包括一项教育和推广计划，旨在提高妇女和未被充分代表的少数民族（URM）在机器人和计算领域的参与和保留，与UMBC庞大的URM人口和该领域的世界级项目合作。该奖项旨在解决人机交互过程中的协作学习和成功表现如何通过学习和基于基础语言来完成。为了实现这一点，该项目将围绕学习抽象知识的结构化表示，并以目标导向的任务完成为基础，以物理环境为基础。有三个高层次的研究重点。首先，将开发新的感知模型来学习机器人的多个异构传感器和数据流之间的对齐。在第二部分中，将开发同步基础语言模型，以更好地捕获完成任务所需的一般语言和隐含上下文期望。第三，将开发一个深度强化学习框架，该框架可以利用前两个重点所取得的进展，从而开发学习概念知识的技术。综上所述，这些进步将允许智能体实现领域适应，改善其在新环境中的行为，并在机器人智能体之间转移概念知识。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project will enable robots to learn to perform tasks with human teammates from language and other human modalities, and then transfer the learned knowledge across heterogeneous platforms and tasks. This will ultimately allow human-robot teaming in domains where people use varied language and instructions to complete complex tasks. As robots become more capable and ubiquitous, they are increasingly moving into complex, human-centric environments such as workplaces and homes. Being able to deploy useful robots in settings where human specialists are stretched thin, such as assistive technology, elder care, and education, has the potential to have far-reaching impacts on human quality of life. Achieving this will require the development of robots that learn, from natural interaction, about an end user's goals and environment. This work is intended to make robots more accessible and usable for non-specialists. In order to verify success and involve the broader community, tasks will be drawn from and tested in conjunction with community Makerspaces, which are strongly linked with both education and community involvement. The award includes an education and outreach plan designed to increase participation by and retention of women and underrepresented minorities (URM) in robotics and computing, engaging with UMBC's large URM population and world-class programs in this space.This award addresses how collaborative learning and successful performance during human-robot interactions can be accomplished by learning from and acting on grounded language. To accomplish this, this project will revolve around learning structured representations of abstract knowledge with goal-directed task completion, grounded in a physical context. There are three high-level research thrusts. In the first, new perceptual models to learn an alignment among a robot's multiple, heterogeneous sensor and data streams will be developed. In the second, synchronous grounded language models will be developed to better capture both general linguistic and implicit contextual expectations that are needed for completing tasks. In the third, a deep reinforcement learning framework will be developed that can leverage the advances achieved by the first two thrusts, allowing the development of techniques for learning conceptual knowledge. Taken together, these advances will allow an agent to achieve domain adaptation, improve its behaviors in new environments, and transfer conceptual knowledge among robotic agents.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.