RI: Small: Model-Based Deep Reinforcement Learning for Domain Transfer

RI：小型：用于域迁移的基于模型的深度强化学习

• 批准号: 1614653
• 负责人: Sergey Levine
• 金额: $ 47.93万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2016-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1614653&HistoricalAwards=false
• 关键词: RI; Small; Model; Based; Deep

The goal of this project is to develop machine learning algorithms that can enable automated decision making and control in applications that require autonomous agents to interact with the real world. In particular, the project will examine two application areas: autonomous robots and educational agents that interact with human students to facilitate learning. The principal technical development investigated in this project will center around applications of deep neural networks (deep learning) to efficiently learn predictive models of the world, such as the physical environment of the robot or the behavior of a human student using an interactive educational agent. Deep learning has enabled impressive advances in passive perception domains such as computer vision and speech recognition, but typically requires very large amounts of data to succeed. This is often a major challenge in interactive settings, where a robot cannot interact with its environment for weeks or months just to learn a single behavior. To address this challenge, this project will investigate how predictive models can be transferred from prior tasks into a new task. The technologies developed as part of this project could enable substantially more sophisticated autonomous systems that can adapt quickly to new situations through transfer. Economic impact could include new consumer robotics products and improved education through intelligent automation.Reinforcement learning holds the promise of automating complex decision making and control in the presence of uncertainty. For a wide range of real-world problems, from robotic control and autonomous vehicles to interactive educational tools, this would provide dramatic improvements in capability and reduction in engineering cost. However, applying reinforcement learning to complex, unstructured environments and real-world problems with raw inputs, such as images and sounds, remains tremendously difficult. Deep learning has shown a great deal of promise for tackling complex learning problems, especially ones that require parsing high-dimensional, raw sensory signals, but the most successful applications of deep learning use very large amounts of labeled data. This is at odds with the demands of reinforcement learning, where the goal is typically to learn an effective policy using the minimal amount of interaction. This projects aims to address this challenge by developing algorithms for model-based deep reinforcement learning, where a generalizable model is learned from past experience on related but different tasks, and then transferred to a new task to learn it very quickly, directly using raw sensory inputs.

该项目的目标是开发机器学习算法，在需要自主代理与现实世界交互的应用程序中实现自动决策和控制。特别是，该项目将研究两个应用领域：自主机器人和与人类学生互动以促进学习的教育代理。本项目研究的主要技术发展将围绕深度神经网络（深度学习）的应用，以有效地学习世界的预测模型，例如机器人的物理环境或使用交互式教育代理的人类学生的行为。深度学习在计算机视觉和语音识别等被动感知领域取得了令人印象深刻的进步，但通常需要非常大量的数据才能成功。在交互式环境中，这通常是一个主要的挑战，因为机器人不能为了学习一种行为而与环境进行数周或数月的交互。为了应对这一挑战，该项目将研究如何将预测模型从先前的任务转移到新任务中。作为该项目的一部分开发的技术可以使更复杂的自主系统能够通过转移迅速适应新情况。经济影响可能包括新的消费机器人产品，以及通过智能自动化改善教育。强化学习有望在存在不确定性的情况下实现复杂决策和控制的自动化。对于广泛的现实问题，从机器人控制和自动驾驶汽车到交互式教育工具，这将大大提高能力并降低工程成本。然而，将强化学习应用于复杂的非结构化环境和具有原始输入（如图像和声音）的现实世界问题仍然非常困难。深度学习在解决复杂的学习问题，特别是那些需要解析高维、原始感官信号的问题上显示出了很大的希望，但深度学习最成功的应用使用了大量的标记数据。这与强化学习的要求不一致，强化学习的目标通常是使用最少的交互来学习有效的策略。该项目旨在通过开发基于模型的深度强化学习算法来解决这一挑战，其中可推广的模型从过去相关但不同的任务的经验中学习，然后转移到新任务中，直接使用原始感官输入快速学习。