RI: Small: Scaling up Robot Learning by Understanding Internet Videos

即使是简单的常识(例如，抽屉可以通过拉动把手打开，或者如何在没有地图的情况下有效地找到去新酒店的咖啡店的路)，也很难大规模地以自动方式纳入控制算法。将这些知识以手工设计的规则的形式结合起来，会导致脆弱的系统，而且不会扩大规模。这就需要使用机器学习来从数据中自动学习这些知识。目前的机器学习技术主要是通过反复试验自己发现这些知识来学习的。这不仅计算成本很高，而且还会导致不适用于新操作条件的专门化行为。这使得部署这样的学习控制算法具有挑战性和单调乏味。与此同时，这种世界知识很容易在互联网上发现的执行不同任务的人的第一和第三人称视频的数据集中得到描述。这个项目将通过开发从这类视频中提取知识的技术来推动最先进的技术，以帮助学习决策和控制算法。该项目开发的技术将使机器人的培训更容易、更快、更好(例如在自动化制造中)。这将使在以前未见过的环境中为基本导航和操作任务更广泛地采用已学习的策略。针对基本机器人任务的有效政策将有助于未来机器人学对更高层次问题(如任务规划和人-机器人交互)的研究，以及对交互问题(如主动学习和主动感知)的计算机视觉研究。该项目还将通过开发专门课程和参与研究，促进研究生和本科生的教育，并通过无障碍传播研究，促进整个研究界。为了从互联网视频中学习机器人政策，该项目将开发一个框架，利用通过直接互动学习和大规模视频理解之间的协同效应，相互学习和借鉴。研究人员将开发视频理解技术，允许a)建立对对象状态敏感的表征，b)获得短程导航和操作技能，以及c)学习用于完成任务的世界知识编码的值函数。总而言之，这些将使得能够很好地推广的控制策略的样本高效学习。研究人员将收集和整理相关数据集。将对这些数据集进行处理，使其能够通过在空间和时间上调整相关视频并将过渡转化为行动来学习有用的政策和表述。通过学习状态敏感特征、参数化技能和获取有关世界的知识的值函数，将提取用于策略学习的有用表示。与现有方法相比，建议框架的有效性将通过更快的学习和更好的学习行为泛化来展示。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Even simple common sense knowledge (for example, that drawers can be opened by pulling on handles, or how to efficiently find the way to a coffee shop in a new hotel without a map) is hard to incorporate into control algorithms in an automated manner at scale. Incorporating such knowledge in the form of hand-designed rules leads to brittle systems and does not scale. This necessitates the use of machine learning to automatically learn such knowledge from data. Current machine learning techniques largely learn by discovering this knowledge by themselves via trial-and-eror. Not only is this computationally expensive, but it also results in specialized behavior that does not generalize to new operating conditions. This makes it challenging and tedious to deploy such learned control algorithms. At the same time, such world knowledge is readily depicted in datasets of first and third-person videos of people conducting different tasks found on the Internet. This project will advance the state-of-the-art by developing techniques to extract knowledge from such videos, to aid learning of decision making and control algorithms. Techniques developed in this project will enable easier, faster, and better training of robots (such as in automated manufacturing). This will enable broader adoption of learned policies for basic navigation and manipulation tasks in previously unseen environments. Effective policies for basic robotic tasks will aid future robotics research on higher-level problems (such as task planning, and human-robot interaction), and computer vision research on interactive problems (like active learning and active perception). The project will also contribute to the education of graduate and undergraduate students by the development of specialized courses and involvement in research, and the research community at large through accessible dissemination of research. In order to learn robot policies from Internet videos, the project will develop a framework that leverages the synergies between learning via direct interaction and large-scale video understanding, to learn from and for each other. Researchers will develop video understanding techniques that allow a) building representations that are sensitive to object states, b) acquiring skills for short-range navigation and manipulation, and c) learning value functions that encode world knowledge for task completion. Collectively, these will enable sample efficient learning of control policies that generalize well. Researchers will collect and curate relevant datasets. These datasets will be processed to make them amenable for learning useful policies and representations, by aligning relevant videos in space and time, and grounding transitions into actions. Useful representations for policy learning will be extracted by learning state-sensitive features, parameterized skills, and value functions that capture knowledge about the world. The effectiveness of the proposed framework will be demonstrated through faster learning and better generalization of learned behaviors as compared to existing approaches.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.