RI: Small: Model-Based Deep Reinforcement Learning for Domain Transfer

该项目的目标是开发机器学习算法，可以在需要自主代理与真实的世界交互的应用程序中实现自动决策和控制。特别是，该项目将研究两个应用领域：自主机器人和与人类学生互动以促进学习的教育代理。该项目研究的主要技术发展将围绕深度神经网络（深度学习）的应用，以有效地学习世界的预测模型，例如机器人的物理环境或使用交互式教育代理的人类学生的行为。深度学习在计算机视觉和语音识别等被动感知领域取得了令人印象深刻的进步，但通常需要非常大量的数据才能取得成功。这通常是交互环境中的一个主要挑战，机器人无法与其环境交互数周或数月，只是为了学习一个单一的行为。为了应对这一挑战，该项目将研究如何将预测模型从先前的任务转移到新的任务中。作为该项目的一部分开发的技术可以使更复杂的自主系统能够通过转移快速适应新情况。经济影响可能包括新的消费机器人产品和通过智能自动化改善教育。强化学习有望在存在不确定性的情况下自动化复杂的决策和控制。对于从机器人控制和自动驾驶汽车到交互式教育工具的各种现实问题，这将大大提高能力并降低工程成本。然而，将强化学习应用于复杂的非结构化环境和具有原始输入（如图像和声音）的现实问题仍然非常困难。深度学习在解决复杂的学习问题方面表现出了很大的潜力，特别是那些需要解析高维原始感觉信号的问题，但深度学习最成功的应用程序使用了大量的标记数据。这与强化学习的要求不一致，强化学习的目标通常是使用最少的交互来学习有效的策略。该项目旨在通过开发基于模型的深度强化学习算法来应对这一挑战，其中从相关但不同任务的过去经验中学习可推广的模型，然后直接使用原始感官输入将其转移到新任务中以快速学习。

The goal of this project is to develop machine learning algorithms that can enable automated decision making and control in applications that require autonomous agents to interact with the real world. In particular, the project will examine two application areas: autonomous robots and educational agents that interact with human students to facilitate learning. The principal technical development investigated in this project will center around applications of deep neural networks (deep learning) to efficiently learn predictive models of the world, such as the physical environment of the robot or the behavior of a human student using an interactive educational agent. Deep learning has enabled impressive advances in passive perception domains such as computer vision and speech recognition, but typically requires very large amounts of data to succeed. This is often a major challenge in interactive settings, where a robot cannot interact with its environment for weeks or months just to learn a single behavior. To address this challenge, this project will investigate how predictive models can be transferred from prior tasks into a new task. The technologies developed as part of this project could enable substantially more sophisticated autonomous systems that can adapt quickly to new situations through transfer. Economic impact could include new consumer robotics products and improved education through intelligent automation.Reinforcement learning holds the promise of automating complex decision making and control in the presence of uncertainty. For a wide range of real-world problems, from robotic control and autonomous vehicles to interactive educational tools, this would provide dramatic improvements in capability and reduction in engineering cost. However, applying reinforcement learning to complex, unstructured environments and real-world problems with raw inputs, such as images and sounds, remains tremendously difficult. Deep learning has shown a great deal of promise for tackling complex learning problems, especially ones that require parsing high-dimensional, raw sensory signals, but the most successful applications of deep learning use very large amounts of labeled data. This is at odds with the demands of reinforcement learning, where the goal is typically to learn an effective policy using the minimal amount of interaction. This projects aims to address this challenge by developing algorithms for model-based deep reinforcement learning, where a generalizable model is learned from past experience on related but different tasks, and then transferred to a new task to learn it very quickly, directly using raw sensory inputs.