NRI: Collaborative Research: Scalable Robot Autonomy through Remote Operator Assistance and Lifelong Learning

在商业应用中广泛采用自主机器人系统的最大障碍之一是在不受限制的人类环境中实现100%可靠的自主性的挑战。通向更强大的自主性的一条途径是花更多的时间在研究实验室中提高机器人的能力，推迟部署，直到自主性完全强大。相反，在野外部署机器人并根据部署后遇到的罕见例子、角落情况和意外情况调整它们的行为，以实现短期、完全可靠的自主可能是有价值的。这种方法特别受到呼叫中心模型的推动，在该模型中，机器人部署在最终用户站点，并在遇到错误时联系远程操作员寻求帮助。该项目开发了一种系统，使机器人能够从远程人工协助中执行终身、渐进的改进，并实现完全自主的长期目标。这项研究计划具有重大的更广泛的影响，使个人机器人更容易为普通人所用，同时也提供了人与机器人互动的机会，这是教育K-12课程以及本科生和研究生教育的理想选择。为了实现这些目标，正在开发新的算法、界面和用户研究，以推进与呼叫中心模型相关的三个关键领域的技术水平：(1)健壮的、多感官的任务结果检测：用于识别寻求帮助或部署恢复行为的条件的多模式技术；(2)用于情境感知的透明设备：用于增加远程操作员的情境感知并允许直观交互的视觉和语言界面模式，导致更有效和正确的恢复过程；(3)改进低级别和高级别任务模型：将纠正和恢复程序纳入现有任务模型的终身学习技术，以及收集更有针对性的数据的积极学习方法。目前正在对酒店礼宾机器人可能执行的各种移动操作任务进行评估，例如送货任务或准备和清理会议宴会后的工作。

One of the most significant barriers to the wider adoption of autonomous robotic systems in commercial applications is the challenge of achieving 100% reliable autonomy in unconstrained human environments. One path toward more robust autonomy is to spend more time in research labs improving robot capabilities, delaying deployment until autonomy is entirely robust. Instead, it may be valuable to deploy robots out in the wild and adapt their behavior based on the rare examples, corner cases, and contingencies encountered after deployment in order to achieve near-term, fully reliable autonomy. This approach is specifically motivated by the call center model, in which robots are deployed at end-user sites and contact a remote human operator for assistance whenever an error is encountered. This project develops a system that enables robots to perform lifelong, incremental improvement from remote human assistance with the long-term goal of achieving full autonomy. This research program has significant broader impacts, making personal robots more accessible to everyday people, while also providing opportunities for human-robot interaction that are ideal for educational K-12 programs, as well as undergraduate and graduate education. Towards these goals, novel algorithms, interfaces, and user studies are being developed to advance the state of the art in three key areas related to the call center model: (1) Robust, Multi-Sensory Task Outcome Detection: multimodal techniques for identifying conditions under which to seek assistance or deploy recovery behaviors; (2) Transparency Devices for Situated Awareness: visual and language interface modalities for increasing the situational awareness of the remote operator and allowing for intuitive interaction, leading to more efficient and correct recovery procedures; (3) Low-Level and High-Level Task Model Refinement: lifelong learning techniques for incorporating corrections and recovery procedures into existing task models, as well as active learning methods to collect more targeted data. The proposed approach is being evaluated on a variety of mobile manipulation tasks that a hotel concierge robot might perform, such as delivery tasks or preparing for and cleaning up after a conference banquet.