Improving Sample Efficiency of Reinforcement Learning

提高强化学习的样本效率

• 批准号: 2579743
• 金额: --
• 依托单位: University of Edinburgh
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2021
• 资助国家: 英国
• 起止时间: 2021 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2579743
• 关键词: Improving; Sample; Efficiency; Reinforcement; Learning

Deep reinforcement learning has had huge empirical success and is a major enabling technology for many applications of AI. However, recent RL algorithms still require millions of samples to obtain good performance. Since obtaining environment interactions is often costly and since challenging environments are rarely static, this inhibits many practical applications. This project will investigate ways of reducing this cost, aiming to find more sample-efficient RL algorithms. We aim for the algorithms to be deployable in realistic settings, where agents use deep networks to represent knowledge about the environment. It is also likely to lead to improved performance of other systems making automated decisions. Research StrategyThe project will investigate two main avenues for improving sample efficiency. Firstly, using a Bayesian framework to gain additional information from samples, we hope to achieve improved exploration which will in turn lead to more informative samples. Secondly, using meta-learning, we hope to enable generalisation, reducing the amount of samples required to learn a task which is similar to other learned tasks.Objectives and ApplicationsWe aim to develop more sample efficient reinforcement learning algorithms and to gain new insights about exploration. The project will be carried out in collaboration with Microsoft Research Cambridge and will have immediate relevance for their computer games research, particularly for training game AI in complex worlds where samples are expensive. The project will have wider applications for any problem which involves decision making with limited data, including real world applications such as robotics and pricing strategies.

深度强化学习已经取得了巨大的经验成功，是人工智能许多应用的主要支持技术。然而，最近的强化学习算法仍然需要数百万个样本才能获得良好的性能。由于获得环境交互通常成本高昂，并且具有挑战性的环境很少是静态的，这抑制了许多实际应用。该项目将研究降低成本的方法，旨在找到样本效率更高的强化学习算法。我们的目标是让算法能够在现实环境中部署，其中代理使用深度网络来表示有关环境的知识。它还可能导致其他系统做出自动化决策的性能提高。研究策略该项目将研究提高样本效率的两个主要途径。首先，使用贝叶斯框架从样本中获取额外信息，我们希望实现改进的探索，从而获得更多信息的样本。其次，使用元学习，我们希望能够实现泛化，减少学习与其他学习任务类似的任务所需的样本量。目标和应用我们的目标是开发更多样本有效的强化学习算法，并获得有关探索的新见解。该项目将与微软剑桥研究院合作开展，并将与其计算机游戏研究直接相关，特别是在样本昂贵的复杂世界中训练游戏人工智能。该项目将对涉及有限数据决策的任何问题有更广泛的应用，包括机器人和定价策略等现实世界的应用。