RI: Small: Efficient Reinforcement Learning for Generic Large-Scale Tasks

自主代理研究的最新进展正在推动我们的社会更接近在日常生活中广泛采用自主代理的边缘。包含代理的应用程序已经存在或正在迅速出现，例如家用机器人、自动驾驶汽车和财务管理代理。序列决策的强化学习（RL）是实现自主智能体广泛部署的重要范例。然而，尽管有一些显著的成功，最先进的强化学习算法还不能完全解决通用的大规模应用。本项目将从四个方向推进强化学习系统的规模化应用。具体来说，该项目是(1)开发算法来自动构建用于学习的输入、输出和策略表示；(2)引入可并行强化学习算法，利用现代并行架构；(3)将抽象和层次推理与基于模型的学习相结合，实现大规模环境的智能探索；(4)使强化学习算法能够从与人类用户的低带宽交互中受益。最后，我们打算将上述四个研究重点统一为一个算法，并对现实世界/大规模应用进行实证评估，包括双足机器人平衡和行走，模拟机器人足球和真实机器人，以及能够在城市环境中规划路径的全尺寸自动驾驶汽车。除了研究进展和对改善国家基础设施的影响外，该项目还将促进本科和研究生课程的发展。

Recent advances in autonomous agents research are pushing our society closer to the brink of the widespread adoption of autonomous agents in everyday life. Applications that incorporate agents already exist or are quickly emerging, such as domestic robots, autonomous vehicles, and financial management agents. Reinforcement learning (RL) of sequential decision making is an important paradigm for enabling the widespread deployment of autonomous agents. However, a few notable successes notwithstanding, state-of-the-art reinforcement learning algorithms are not yet fully capable of addressing generic large-scale applications. This project is advancing in four directions to scale-up application of RL systems. Specifically, the project is (1) developing algorithms to automatically structure the input, output, and policy representations for learning; (2) introducing parallelizable reinforcement learning algorithms so as to exploit modern parallel architectures; (3) unifying abstraction and hierarchical reasoning with model-based learning for the purpose of enabling intelligent exploration of large-scale environments; and (4) enabling reinforcement learning algorithms to benefit from low-bandwidth interactions with human users. Finally, we intend to unify the four research thrusts above into a single algorithm and conduct empirical evaluation on real-world/large-scale applications, to include biped robot balancing and walking, robot soccer in simulation and with real robots, and a full-size autonomous vehicle capable of planning paths in an urban environment.In addition to research advances and implications for improving national infrastructure, the project will contribute to undergraduate and graduate curriculum development.