CAREER: When Reality Fails Expectations: Containing Reflective Domain Models for Human-Aware Planning and Learning of Robotic Teammates

这一学院早期职业发展(Career)计划将引入一种被称为反射机器人的变革性范式，用于规划和学习机器人队友。尽管对自主机器人代理的发展进行了大量研究，但人类仍然对他们的机器人队友感到矛盾。问题不仅在于机器人与人类互动的局限性，还在于人类无法理解他们的机器人伙伴。人类往往对机器人过于乐观或悲观，导致现实和期望之间不可饶恕的不协调，最终导致严重的团队失败。这项研究将解决由于人类和机器人对任务领域的不同理解而导致这种不一致的根本原因。在这种情况下，对机器人来说，了解如何协调差异以保持适当的团队合作是至关重要的。扩大机器人技术适用性的框架与合作机器人发展的广泛呼声相一致。它代表了可解释和安全的机器人在人与机器人交互中的关键推动因素。教育目标是创新机器人教育以激发和吸引学生，培训下一代科学家和工程师进行人-机器人合作，让公众参与讨论，并提高对机器人技术的信任。这些活动将使K-12学生、本科生和研究生以及代表性不足的学生受益。具体的研究目标是通过开发模型反思规划和学习方法来解决人类和机器人之间对任务领域的不同理解，称为反射域模型，以形成反射机器人的理论和算法基础。包含这种反射域模型的关键是机器人维护对真实域模型和人类对它的理解的估计，并利用这两者来通知其操作。具体地说，1)模型反思规划对规划范式的转变做出了独特的贡献，将规划方法推广到现实世界应用的开放领域。与仅依赖单一领域模型的传统规划方法不同，模型反射规划还考虑了人类对它的理解。贝叶斯方法是积极地为这种理解建模，同时将分层信息合并为状态和动作抽象，以便为规划方法提供信息。因此，模型反思规划有助于实现机器人队友在现实世界中的自主性。2)模型反思性学习倡导学习方法设计的范式转换，以推广到具有反思性模型的情况，这是正常的，而不是非专家用户的例外。研究了一种基于熵增广强化学习与变分推理相结合的学习框架。它通过解决现有学习系统中的一个关键漏洞来避免陷阱，在现有的学习系统中，人类用户可能会以非琐碎和系统的方式误导学习。同时，这些成果将为国际机器人协会的长期机器人职业生涯提供令人振奋的新基础，并在社区中创造新的增长和领导机会。该项目由跨部门机器人基础研究计划支持，该计划由工程指导委员会(ENG)和计算机和信息科学与工程指导委员会(CEISE)联合管理和资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) program will introduce a transformative paradigm, referred to as Reflective Robotics, for planning and learning of robotic teammates. Despite significant research into the development of autonomous robotic agents, humans are still experiencing ambivalence towards their robotic teammates. The problem lies not only in the limitations of robots to interacting with humans, but also in the humans’ inability to understand their robotic partners. Humans tend to be overly optimistic or pessimistic towards robots, resulting in an unforgiving discordance between reality and expectations and, ultimately, egregious teaming failures. This study will address a fundamental cause for such a discordance due to different understandings of the task domain between the human and the robot. In such situations, it is crucial for the robot to understand how to reconcile the discrepancy to maintain proper teaming. The framework to expand the applicability of robotic technologies aligns with the broad call for co-robot development. It represents a key enabler of interpretable and safe robots for human-robot interaction. The educational goal is to innovate robotics education to excite and attract students, train the next generation of scientists and engineers in human-robot collaboration, engage the public audience in the discussion, and boost trust in robotics technologies. The activities will benefit K-12 students, undergraduate and graduate students, and students from underrepresented groups.The specific research goal is to address the different understandings of the task domain between the human and the robot, referred to as reflective domain models, by developing model reflective planning and learning methods to form the theoretical and algorithmic foundation of Reflective Robotics. The key to containing such reflective domain models is for the robot to maintain estimates of the true domain model and the human’s understanding of it and utilize both to inform its operation. In particular, 1) Model Reflective Planning contributes uniquely to the paradigm shift in planning to generalize planning methods to open-world domains for real-world applications. In contrast to the traditional planning methods that depend only on a single domain model, model reflective planning also considers the human’s understanding of it. A Bayesian approach is to actively model such an understanding while incorporating the hierarchical information as state and action abstractions to inform planning methods. As such, model reflective planning contributes to the realization of real-world autonomy for robotic teammates. 2) Model Reflective Learning advocates a paradigm shift in the design of learning methods to generalize to situations with reflective models, which are the norm rather than the exceptions with non-expert users. A framework based on entropy augmented reinforcement learning integrated with variational inference will be investigated. It avoids a pitfall by addressing a critical gap in the existing learning systems where human users can mislead learning in non-trivial and systematic ways. At the same time, these results will provide an exciting new foundation for the PI’s long-term career in robotics and generate new growth and leadership opportunities in the community.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.