CAREER: When Reality Fails Expectations: Containing Reflective Domain Models for Human-Aware Planning and Learning of Robotic Teammates

这个教师早期职业发展（Career）项目将引入一种被称为反射机器人的变革范式，用于规划和学习机器人队友。尽管在自主机器人代理的发展方面进行了大量的研究，但人类对他们的机器人队友仍然存在矛盾心理。问题不仅在于机器人与人类互动的局限性，还在于人类无法理解他们的机器人伙伴。人类往往对机器人过于乐观或悲观，导致现实与期望之间出现不可原谅的不一致，最终导致令人震惊的团队失败。这项研究将解决由于人类和机器人之间对任务域的不同理解而导致这种不一致的根本原因。在这种情况下，机器人了解如何调和差异以保持适当的团队是至关重要的。扩大机器人技术适用性的框架与协作机器人发展的广泛要求是一致的。它代表了人机交互中可解释和安全的机器人的关键使能器。教育目标是创新机器人教育，以激发和吸引学生，培养下一代科学家和工程师进行人机协作，让公众参与讨论，并提高对机器人技术的信任。这些活动将使K-12学生、本科生和研究生以及来自代表性不足群体的学生受益。具体的研究目标是通过开发模型反射规划和学习方法，解决人与机器人之间对任务域的不同理解，称为反射域模型，形成反射机器人的理论和算法基础。包含这种反射域模型的关键是让机器人保持对真实域模型的估计和人类对它的理解，并利用这两者来告知它的操作。特别是，1)模型反思性规划对规划的范式转变做出了独特的贡献，将规划方法推广到现实世界应用的开放世界领域。与传统的仅依赖于单一领域模型的规划方法不同，模型反射规划还考虑了人对它的理解。贝叶斯方法是积极地为这种理解建模，同时将层次信息作为状态和动作抽象合并到规划方法中。因此，模型反射规划有助于实现机器人队友的现实世界自治。2)模型反思性学习提倡在学习方法的设计中进行范式转换，以推广到具有反思性模型的情况，这是非专家用户的常态而不是例外。研究了一种基于变分推理的熵增强强化学习框架。它通过解决现有学习系统中人类用户可能以非琐碎和系统的方式误导学习的关键漏洞，避免了一个陷阱。同时，这些结果将为PI在机器人领域的长期职业生涯提供令人兴奋的新基础，并在社区中产生新的增长和领导机会。该项目由跨部门机器人基础研究项目支持，由工程（ENG）和计算机与信息科学与工程（CISE）联合管理和资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This Faculty Early Career Development (CAREER) program will introduce a transformative paradigm, referred to as Reflective Robotics, for planning and learning of robotic teammates. Despite significant research into the development of autonomous robotic agents, humans are still experiencing ambivalence towards their robotic teammates. The problem lies not only in the limitations of robots to interacting with humans, but also in the humans’ inability to understand their robotic partners. Humans tend to be overly optimistic or pessimistic towards robots, resulting in an unforgiving discordance between reality and expectations and, ultimately, egregious teaming failures. This study will address a fundamental cause for such a discordance due to different understandings of the task domain between the human and the robot. In such situations, it is crucial for the robot to understand how to reconcile the discrepancy to maintain proper teaming. The framework to expand the applicability of robotic technologies aligns with the broad call for co-robot development. It represents a key enabler of interpretable and safe robots for human-robot interaction. The educational goal is to innovate robotics education to excite and attract students, train the next generation of scientists and engineers in human-robot collaboration, engage the public audience in the discussion, and boost trust in robotics technologies. The activities will benefit K-12 students, undergraduate and graduate students, and students from underrepresented groups.The specific research goal is to address the different understandings of the task domain between the human and the robot, referred to as reflective domain models, by developing model reflective planning and learning methods to form the theoretical and algorithmic foundation of Reflective Robotics. The key to containing such reflective domain models is for the robot to maintain estimates of the true domain model and the human’s understanding of it and utilize both to inform its operation. In particular, 1) Model Reflective Planning contributes uniquely to the paradigm shift in planning to generalize planning methods to open-world domains for real-world applications. In contrast to the traditional planning methods that depend only on a single domain model, model reflective planning also considers the human’s understanding of it. A Bayesian approach is to actively model such an understanding while incorporating the hierarchical information as state and action abstractions to inform planning methods. As such, model reflective planning contributes to the realization of real-world autonomy for robotic teammates. 2) Model Reflective Learning advocates a paradigm shift in the design of learning methods to generalize to situations with reflective models, which are the norm rather than the exceptions with non-expert users. A framework based on entropy augmented reinforcement learning integrated with variational inference will be investigated. It avoids a pitfall by addressing a critical gap in the existing learning systems where human users can mislead learning in non-trivial and systematic ways. At the same time, these results will provide an exciting new foundation for the PI’s long-term career in robotics and generate new growth and leadership opportunities in the community.This project is supported by the cross-directorate Foundational Research in Robotics program, jointly managed and funded by the Directorates for Engineering (ENG) and Computer and Information Science and Engineering (CISE).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.