CRII: RI: Active Learning of Preferences for Human-Aware Autonomy

人类的偏好在确定机器人系统应该如何行动方面发挥着关键作用，例如，辅助机器人手臂应该如何移动，或者自动驾驶汽车应该如何驾驶。学习到的人类偏好是交互式自主系统规划中的一个重要元素，例如，机器人与不同类型的人类队友协作，或与人类共享自主权以有效地远程操作机器人手臂。人们希望学习技术可以用来学习代表人类对机器人应用偏好的奖励函数。然而，学习算法成功的一个重要部分可以归因于大量标记数据的可用性。不幸的是，在机器人应用中收集和标记数据既昂贵又耗时。此外，人类并不总是能够可靠地为给定的机器人动作分配成功值（奖励），并且由于操作具有多个自由度的机器人的难度，他们的演示通常是次优的。提出的研究开发了基础技术，以解决在人机交互中使用学习技术的关键挑战。该项目的目标是开发有效的方法和算法，以便在与机器人操作、交互和协作时更好地建模和理解人类的偏好。此外，研究人员还将设计算法，让机器人意识到这种偏好，并能与人类进行安全无缝的互动。该项目涉及两个主要贡献：(1)开发高效和主动的算法来学习基于比较和排名的人类偏好的概率混合模型。(2)开发利用人类偏好的机器人规划算法，以实现无缝共享自主权和更有效的人机交互。在自动驾驶领域的初步结果表明，人们可以学习人类的驾驶偏好，这种方法可以提高机器人系统的效率和安全性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Humans' preferences play a key role in specifying how robotics systems should act, i.e., how an assistive robot arm should move, or how an autonomous car should drive. The learned human preferences are an important element in planning for interactive autonomous systems, e.g., robots collaborating with different types of human teammates, or shared autonomy with a human to efficiently teleoperate a robot arm. One hopes that learning techniques can be used to learn reward functions representing humans' preferences for robotics applications. However, a significant part of the success of learning algorithms can be attributed to the availability of large amounts of labeled data. Unfortunately, collecting and labeling data can be costly and time-consuming in robotics applications. In addition, humans are not always capable of reliably assigning a success value (reward) to a given robot action, and their demonstrations are usually suboptimal due to the difficulty of operating robots with more than a few degrees of freedom. The proposed research develops foundational techniques to address the key challenges of using learning techniques in human-robot interaction. The goal of this project is to develop efficient methods and algorithms to first better model and understand humans preferences while operating, interacting, and collaborating with robots. Furthermore, the investigator will design algorithms that plan for robots that are aware of such preferences and can initiate a safe and seamless interaction with humans. This project involves two main contributions: (1) Developing efficient and active algorithms to learn probabilistic mixture models for humans preferences about how a robot should operate based on comparisons and rankings., and (2) Developing planning algorithms for robots that leverage humans preferences to enable seamless shared autonomy and more efficient human-robot interaction. Preliminary results in the domain of autonomous driving suggest that one can learn driving preferences of humans and this approach can improve efficiency and safety of robotics systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.