NRI: FND: Semi-Supervised Deep Learning for Domain Adaptation in Robotic Language Acquisition

该项目将使机器人能够从语言和其他人类形态中学习与人类队友一起执行任务，然后在不同的平台和任务之间传递所学到的知识。这最终将允许人和机器人在人们使用不同语言和指令完成复杂任务的领域进行合作。随着机器人变得更有能力和无处不在，它们越来越多地进入工作场所和家庭等以人为中心的复杂环境中。能够在人类专家力不从心的环境中部署有用的机器人，如辅助技术、老年人护理和教育，可能会对人类的生活质量产生深远影响。要实现这一点，将需要开发能够从自然互动中学习最终用户的目标和环境的机器人。这项工作的目的是让非专业人士更容易接触和使用机器人。为了验证成功并使更广泛的社区参与进来，将从与教育和社区参与密切相关的社区创建者空间中抽取任务并与之一起进行测试。该奖项包括一个教育和推广计划，旨在增加女性和未被充分代表的少数群体(URM)在机器人和计算机领域的参与和留住，参与UMBC在该领域的大量URM人口和世界级项目。该奖项旨在说明如何通过学习扎根的语言和行动来实现协作学习和在人与机器人互动过程中的成功表现。为了做到这一点，这个项目将围绕着学习抽象知识的结构化表示，并以物理环境为基础，以目标为导向完成任务。有三个高水平的研究推动力。首先，将开发新的感知模型，以学习机器人多个、不同种类的传感器和数据流之间的比对。在第二个阶段，将开发同步的扎根语言模型，以更好地捕捉完成任务所需的一般语言期望和隐含的语境期望。在第三个阶段，将开发一个深度强化学习框架，该框架可以利用前两个推进所取得的进展，从而开发学习概念知识的技术。综上所述，这些进步将使代理能够实现领域适应，改善其在新环境中的行为，并在机器人代理之间转移概念知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project will enable robots to learn to perform tasks with human teammates from language and other human modalities, and then transfer the learned knowledge across heterogeneous platforms and tasks. This will ultimately allow human-robot teaming in domains where people use varied language and instructions to complete complex tasks. As robots become more capable and ubiquitous, they are increasingly moving into complex, human-centric environments such as workplaces and homes. Being able to deploy useful robots in settings where human specialists are stretched thin, such as assistive technology, elder care, and education, has the potential to have far-reaching impacts on human quality of life. Achieving this will require the development of robots that learn, from natural interaction, about an end user's goals and environment. This work is intended to make robots more accessible and usable for non-specialists. In order to verify success and involve the broader community, tasks will be drawn from and tested in conjunction with community Makerspaces, which are strongly linked with both education and community involvement. The award includes an education and outreach plan designed to increase participation by and retention of women and underrepresented minorities (URM) in robotics and computing, engaging with UMBC's large URM population and world-class programs in this space.This award addresses how collaborative learning and successful performance during human-robot interactions can be accomplished by learning from and acting on grounded language. To accomplish this, this project will revolve around learning structured representations of abstract knowledge with goal-directed task completion, grounded in a physical context. There are three high-level research thrusts. In the first, new perceptual models to learn an alignment among a robot's multiple, heterogeneous sensor and data streams will be developed. In the second, synchronous grounded language models will be developed to better capture both general linguistic and implicit contextual expectations that are needed for completing tasks. In the third, a deep reinforcement learning framework will be developed that can leverage the advances achieved by the first two thrusts, allowing the development of techniques for learning conceptual knowledge. Taken together, these advances will allow an agent to achieve domain adaptation, improve its behaviors in new environments, and transfer conceptual knowledge among robotic agents.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.