Monte Carlo Tree Search with Robust Exploration
Monte Carlo Tree Search with Robust Exploration
复制标题
具有稳健探索的蒙特卡罗树搜索
DOI：
10.1007/978-3-319-50935-8_4
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
T. Imagawa and T. Kaneko
中科院分区：
文献类型：
--
作者：
山田亮太;松谷悠佑;木村尭朗;伊達広行;Takahisa Imagawa and Tomoyuki Kaneko;今川 孝久，金子知適;T. Imagawa and T. Kaneko
本文提出了一种新的蒙特-卡罗树搜索方法，重点是确定最佳移动。UCT最大限度地减少了累积的遗憾，在围棋等游戏中取得了显著的成功。然而，最近对简单后悔的研究表明，有更好的探索策略。为了进一步提高性能，要探索的叶子不仅由平均值决定，而且由整个奖励分布决定。我们采用了一种混合的方法来获得可靠的分布。在搜索树的较浅的一半中使用Negamax风格的奖励分布备份，并且在树的其余部分中采用UCT。在合成树上的实验表明，除了树的宽度和深度均匀外，该方法的性能优于UCT和类似方法。
This paper presents a new Monte-Carlo tree search method that focuses on identifying the best move. UCT which minimizes the cumulative regret, has achieved remarkable success in Go and other games. However, recent studies on simple regret reveal that there are better exploration strategies. To further improve the performance, a leaf to be explored is determined not only by the mean but also by the whole reward distribution. We adopted a hybrid approach to obtain reliable distributions. A negamax-style backup of reward distributions is used in the shallower half of a search tree, and UCT is adopted in the rest of the tree. Experiments on synthetic trees show that this presented method outperformed UCT and similar methods, except for trees having uniform width and depth.