Deep Bayesian Reinforcement Learning in Changing Environments

深度强化学习（DRL）在许多具有固定环境的游戏中表现良好，例如围棋和星际争霸。但大多数现实世界的环境会随着时间的推移而变化，并受到天气等随机因素的影响。因此，DRL环境的非平稳性需要引起更多的关注。这项博士研究揭示了贝叶斯方法在不断变化的环境中的强化学习，因为贝叶斯DRL可以处理具有不确定性的环境。我们所需要做的就是将环境变化视为不确定性。总的来说，我计划追求四个研究方向，以帮助贝叶斯强化学习在不断变化的环境中快速学习。我首先调查利用先前的经验，以确定当前的环境状态在不断变化的环境设置，然后学习策略。然后我建议在贝叶斯强化学习中使用元学习技巧来促进对新环境的适应。最后，我建议在不断变化的环境中使用贝叶斯方法进行不那么保守的稳健RL和更有效的安全RL。这些贝叶斯强化学习方向有助于在不断变化的环境中实现高效、安全和鲁棒的深度贝叶斯强化学习。

Deep Reinforcement Learning (DRL) worked well in a wide range of games with a fixed environment, such as Go and Starcraft. But most real-world environments change over time and are influenced by random factors such as weather. So, the nonstationarity of the environment in DRL requires more attention. This PhD research sheds light on Reinforcement Learning in changing environments with Bayesian approach since Bayesian DRL can deal with environments with uncertainty. All we need is to treat the environmental changes as uncertainty. In general, I plan to pursue four research directions to help fast learning in changing environments with Bayesian RL. I first investigate utilising prior experience to identify current environment status in a changing environment setting and then learn strategies. Then I suggest using the meta-learning tricks to facilitate adaptation to new environments in Bayesian RL. Finally, I propose less conservative Robust RL and more efficient Safe RL in changing environments with a Bayesian approach. These Bayesian RL directions can contribute to an efficient, safe, and robust deep Bayesian RL in changing environments.