CRII: RI: Active Learning of Preferences for Human-Aware Autonomy

人类的偏好在指定机器人系统应该如何行动方面起着关键作用，即，辅助机器人手臂应该如何移动，或者自动驾驶汽车应该如何驾驶。学习到的人类偏好是规划交互式自主系统的重要元素，例如，机器人与不同类型的人类队友合作，或者与人类共享自主权以有效地远程操作机器人手臂。人们希望学习技术可以用于学习代表人类对机器人应用的偏好的奖励函数。然而，学习算法的成功很大一部分可以归因于大量标记数据的可用性。不幸的是，在机器人应用中，收集和标记数据可能是昂贵和耗时的。此外，人类并不总是能够可靠地为给定的机器人动作分配成功值（奖励），并且由于操作具有多个自由度的机器人的困难，他们的演示通常是次优的。拟议的研究开发基础技术，以解决在人机交互中使用学习技术的关键挑战。该项目的目标是开发有效的方法和算法，首先更好地建模和理解人类的偏好，同时与机器人进行操作，交互和协作。此外，研究人员将设计算法，为机器人制定计划，使其能够意识到这些偏好，并能够与人类进行安全和无缝的互动。该项目包括两个主要贡献：（1）开发高效和主动的算法，以学习人类偏好的概率混合模型，即基于比较和排名的机器人应该如何操作。以及（2）开发利用人类偏好的机器人规划算法，以实现无缝共享自治和更有效的人机交互。自动驾驶领域的初步成果表明，人们可以学习人类的驾驶偏好，这种方法可以提高机器人系统的效率和安全性。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Humans' preferences play a key role in specifying how robotics systems should act, i.e., how an assistive robot arm should move, or how an autonomous car should drive. The learned human preferences are an important element in planning for interactive autonomous systems, e.g., robots collaborating with different types of human teammates, or shared autonomy with a human to efficiently teleoperate a robot arm. One hopes that learning techniques can be used to learn reward functions representing humans' preferences for robotics applications. However, a significant part of the success of learning algorithms can be attributed to the availability of large amounts of labeled data. Unfortunately, collecting and labeling data can be costly and time-consuming in robotics applications. In addition, humans are not always capable of reliably assigning a success value (reward) to a given robot action, and their demonstrations are usually suboptimal due to the difficulty of operating robots with more than a few degrees of freedom. The proposed research develops foundational techniques to address the key challenges of using learning techniques in human-robot interaction. The goal of this project is to develop efficient methods and algorithms to first better model and understand humans preferences while operating, interacting, and collaborating with robots. Furthermore, the investigator will design algorithms that plan for robots that are aware of such preferences and can initiate a safe and seamless interaction with humans. This project involves two main contributions: (1) Developing efficient and active algorithms to learn probabilistic mixture models for humans preferences about how a robot should operate based on comparisons and rankings., and (2) Developing planning algorithms for robots that leverage humans preferences to enable seamless shared autonomy and more efficient human-robot interaction. Preliminary results in the domain of autonomous driving suggest that one can learn driving preferences of humans and this approach can improve efficiency and safety of robotics systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.