NRI: Collaborative Research: Scalable Robot Autonomy through Remote Operator Assistance and Lifelong Learning

在商业应用中广泛采用自主机器人系统的最大障碍之一是在不受约束的人类环境中实现100%可靠的自主性的挑战。实现更强大的自主性的一个途径是在实验室里花更多的时间来提高机器人的能力，推迟部署，直到自主性完全强大。相反，将机器人部署到野外，并根据罕见的例子、角落案例和部署后遇到的突发事件来调整它们的行为，以实现近期完全可靠的自主，这可能是有价值的。这种方法特别受到呼叫中心模型的推动，在呼叫中心模型中，机器人部署在最终用户站点，并在遇到错误时联系远程人工操作员寻求帮助。这个项目开发了一个系统，使机器人能够完成终身的、渐进的改进，从远程的人类协助到实现完全自主的长期目标。该研究项目具有显著的更广泛的影响，使个人机器人更容易接触到日常的人，同时也为人机交互提供了机会，这是理想的教育K-12课程，以及本科和研究生教育。为了实现这些目标，新的算法、界面和用户研究正在开发中，以推进与呼叫中心模型相关的三个关键领域的最新技术：(1)鲁棒、多感官任务结果检测：用于识别寻求援助或部署恢复行为的条件的多模式技术；(2)位置感知透明装置：视觉和语言界面模式，用于增加远程操作员的态势感知，并允许直观的交互，从而导致更有效和正确的恢复程序；(3)低级和高级任务模型改进：将修正和恢复过程纳入现有任务模型的终身学习技术，以及收集更有针对性数据的主动学习方法。该方法正在对酒店礼宾机器人可能执行的各种移动操作任务进行评估，例如送货任务或会议宴会后的准备和清理。

One of the most significant barriers to the wider adoption of autonomous robotic systems in commercial applications is the challenge of achieving 100% reliable autonomy in unconstrained human environments. One path toward more robust autonomy is to spend more time in research labs improving robot capabilities, delaying deployment until autonomy is entirely robust. Instead, it may be valuable to deploy robots out in the wild and adapt their behavior based on the rare examples, corner cases, and contingencies encountered after deployment in order to achieve near-term, fully reliable autonomy. This approach is specifically motivated by the call center model, in which robots are deployed at end-user sites and contact a remote human operator for assistance whenever an error is encountered. This project develops a system that enables robots to perform lifelong, incremental improvement from remote human assistance with the long-term goal of achieving full autonomy. This research program has significant broader impacts, making personal robots more accessible to everyday people, while also providing opportunities for human-robot interaction that are ideal for educational K-12 programs, as well as undergraduate and graduate education. Towards these goals, novel algorithms, interfaces, and user studies are being developed to advance the state of the art in three key areas related to the call center model: (1) Robust, Multi-Sensory Task Outcome Detection: multimodal techniques for identifying conditions under which to seek assistance or deploy recovery behaviors; (2) Transparency Devices for Situated Awareness: visual and language interface modalities for increasing the situational awareness of the remote operator and allowing for intuitive interaction, leading to more efficient and correct recovery procedures; (3) Low-Level and High-Level Task Model Refinement: lifelong learning techniques for incorporating corrections and recovery procedures into existing task models, as well as active learning methods to collect more targeted data. The proposed approach is being evaluated on a variety of mobile manipulation tasks that a hotel concierge robot might perform, such as delivery tasks or preparing for and cleaning up after a conference banquet.