RI: Small: Scaling up Robot Learning by Understanding Internet Videos

即使是简单的常识（例如，抽屉可以通过拉动把手打开，或者如何在没有地图的情况下有效地找到去新酒店咖啡店的路）也很难以自动化的方式大规模地整合到控制算法中。将这些知识以手工设计规则的形式结合在一起，会导致系统变得脆弱，并且无法扩展。这就需要使用机器学习来从数据中自动学习这些知识。目前的机器学习技术主要是通过反复试验来发现这些知识。这不仅在计算上很昂贵，而且还会导致无法推广到新的操作条件的特殊行为。这使得部署这种学习控制算法变得具有挑战性和乏味。与此同时，这种世界知识很容易在互联网上找到的第一人称和第三人称视频数据集中描述出来。该项目将通过开发从此类视频中提取知识的技术来推进最先进的技术，以帮助学习决策和控制算法。在这个项目中开发的技术将使机器人的训练更容易、更快、更好（如自动化制造）。这将使在以前看不见的环境中更广泛地采用学习策略来完成基本的导航和操作任务。针对机器人基本任务的有效策略将有助于未来机器人对更高层次问题（如任务规划和人机交互）的研究，以及对交互问题（如主动学习和主动感知）的计算机视觉研究。该项目还将通过开设专门课程和参与研究对研究生和本科生的教育作出贡献，并通过传播研究成果对整个研究界作出贡献。为了从互联网视频中学习机器人政策，该项目将开发一个框架，利用通过直接互动学习和大规模视频理解之间的协同作用，相互学习并为彼此学习。研究人员将开发视频理解技术，允许a)建立对对象状态敏感的表示，b)获得短程导航和操作的技能，以及c)学习编码世界知识以完成任务的价值函数。总的来说，这些将使控制策略的样本有效学习具有良好的泛化性。研究人员将收集和整理相关数据集。通过在空间和时间上对齐相关视频，并将过渡转化为行动，对这些数据集进行处理，使其适合于学习有用的策略和表示。通过学习状态敏感特征、参数化技能和获取世界知识的价值函数，可以提取出对策略学习有用的表示。与现有方法相比，所提出的框架的有效性将通过更快的学习和更好的学习行为泛化来证明。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Even simple common sense knowledge (for example, that drawers can be opened by pulling on handles, or how to efficiently find the way to a coffee shop in a new hotel without a map) is hard to incorporate into control algorithms in an automated manner at scale. Incorporating such knowledge in the form of hand-designed rules leads to brittle systems and does not scale. This necessitates the use of machine learning to automatically learn such knowledge from data. Current machine learning techniques largely learn by discovering this knowledge by themselves via trial-and-eror. Not only is this computationally expensive, but it also results in specialized behavior that does not generalize to new operating conditions. This makes it challenging and tedious to deploy such learned control algorithms. At the same time, such world knowledge is readily depicted in datasets of first and third-person videos of people conducting different tasks found on the Internet. This project will advance the state-of-the-art by developing techniques to extract knowledge from such videos, to aid learning of decision making and control algorithms. Techniques developed in this project will enable easier, faster, and better training of robots (such as in automated manufacturing). This will enable broader adoption of learned policies for basic navigation and manipulation tasks in previously unseen environments. Effective policies for basic robotic tasks will aid future robotics research on higher-level problems (such as task planning, and human-robot interaction), and computer vision research on interactive problems (like active learning and active perception). The project will also contribute to the education of graduate and undergraduate students by the development of specialized courses and involvement in research, and the research community at large through accessible dissemination of research. In order to learn robot policies from Internet videos, the project will develop a framework that leverages the synergies between learning via direct interaction and large-scale video understanding, to learn from and for each other. Researchers will develop video understanding techniques that allow a) building representations that are sensitive to object states, b) acquiring skills for short-range navigation and manipulation, and c) learning value functions that encode world knowledge for task completion. Collectively, these will enable sample efficient learning of control policies that generalize well. Researchers will collect and curate relevant datasets. These datasets will be processed to make them amenable for learning useful policies and representations, by aligning relevant videos in space and time, and grounding transitions into actions. Useful representations for policy learning will be extracted by learning state-sensitive features, parameterized skills, and value functions that capture knowledge about the world. The effectiveness of the proposed framework will be demonstrated through faster learning and better generalization of learned behaviors as compared to existing approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.