Reinforcement Learning Algorithms Designed to Persist

点击翻译按钮获取中文摘要

The field of reinforcement learning is concerned with understanding how intelligent agents can, from trial and error, learn to make decisions that lead to the best outcomes. In silico, its techniques have been applied to wide range of domains, producing computer programs that surpass the world's human champions at the game of Go (2016), can autonomously navigate balloons in the stratosphere (2020), and can design electronics in a fraction of the time taken by human experts (2021). To achieve this level of performance, however, these programs require weeks or even months or training. This is because the most effective reinforcement learning methods are designed to learn to solve a given task from scratch. Using present algorithms it is difficult, if not downright impossible to carry over the learnings from one version of the program to the next. This makes it hard, for example, to support a learning system that evolves and learns over a period of years - a common scenario in practical applications, where a research and development team might continue to improve the learning software over time. The research in this proposal aims to address this shortcoming by studying methods and principles with which previously-acquired experience may be carried across iterations of a learning system. Doing so requires understanding how an agent's immediate experience can be synthesized into a more permanent form called a representation of state, and also how an agent can purposefully act to acquire new information that helps it gains a better understanding of its environment. Fundamental advances in this direction will make it possible to design learning systems that benefit from years, if not decades of experience and can therefore make substantially better decisions.