Computational Theory of Inductive Reinforcement Learning-Bayesian Inference on Environment Search and Inductive Reconstruction

本文主要研究基于贝叶斯推理技术的强化学习环境模型重构问题。在强化学习中，智能体通过反复试验学习环境模型，如果有一个合适的贝叶斯环境模型来表示环境中的不确定性，就可以实现最优探索。为此，我们提出了一种基于预测状态表示的环境描述框架TD-Network的改进方法。此外，我们对隐马尔可夫模型的非参数贝叶斯模型进行了扩展，以表示隐藏状态的层次聚类。此外，我们应用学徒式学习的框架，提出了一种基于贝叶斯推理的从他人行为中构建环境模型的方法。这些都是环境搜索和重建过程的贝叶斯重建所必需的要素。

This study focuses on environmental model reconstruction in reinforcement learning based on Bayesian inference techniques. In reinforcement learning, an agent learns environment model by trial-and-error; if we have a suitable Bayesian environment model that represents uncertainty in the environment, an optimal exploration can be achieved. For this purpose, we proposed new approaches that improve TD-network, an environment description framework based on predictive state representation. In addition, we extended a nonparametric Bayesian model for hidden Markov model to represent hierarchical clustering of hidden states. Moreover, we applied the framework of apprenticeship learning and proposed a method that constructs environment model from other’s actions based on Bayesian inference. These are elements that are required for Bayesian reconstruction of the process of environmental search and reconstruction.