Predicting Opponent Moves for Improving Hearthstone AI
Predicting Opponent Moves for Improving Hearthstone AI
复制标题
预测对手的行动以改进《炉石传说》人工智能
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
R. Kruse
中科院分区：
文献类型：
--
作者：
Alexander Dockhorn;Max Frick;Ünal Akkaya;R. Kruse
游戏为人工智能代理的发展提出了许多有趣的问题。尤其流行的是指导自主代理决策过程的方法，该代理的任务是玩特定的游戏。在以往的研究中，启发式搜索方法蒙特卡洛树搜索(MCTS)被成功地应用于广泛的游戏。结果表明，这种方法往往可以达到与人类一样的演奏能力，甚至更好。然而，收藏性纸牌游戏的特点，如网络游戏炉石，使得直接应用MCTS是不可行的。对手手牌、抽牌和随机牌效果的不确定性很大程度上限制了MCTS的模拟深度。我们表明，从人类回放数据库中收集的知识通过预测多张牌分布来帮助克服这个问题。这些预测可用于增加MCTS的模拟深度。为此，我们计算频繁出现的牌的二元率来预测对手的多套手牌。这些预测可用于创建MCTS代理的集合，这些代理在不同的卡分布的假设下工作，并根据其分配的分布执行模拟。建议的集成方法比游戏中的其他代理性能更好，包括各种类型的MCT。我们的案例研究表明，使用足够准确的预测可以有效地处理不确定性，最终改善MCTS指导的决策过程。由此得到的基于这种MCTS集成的决策被证明不太容易因不确定性而出错，并开辟了一类新的MCTS算法。
Games pose many interesting questions for the development of artificial intelligence agents. Especially popular are methods that guide the decision-making process of an autonomous agent, which is tasked to play a certain game. In previous studies, the heuristic search method Monte Carlo Tree Search (MCTS) was successfully applied to a wide range of games. Results showed that this method can often reach playing capabilities on par with humans or even better. However, the characteristics of collectible card games such as the online game Hearthstone make it infeasible to apply MCTS directly. Uncertainty in the opponent’s hand cards, the card draw, and random card effects considerably restrict the simulation depth of MCTS. We show that knowledge gathered from a database of human replays help to overcome this problem by predicting multiple card distributions. Those predictions can be used to increase the simulation depth of MCTS. For this purpose, we calculate bigram-rates of frequently co-occurring cards to predict multiple sets of hand cards for our opponent. Those predictions can be used to create an ensemble of MCTS agents, which work under the assumption of differing card distributions and perform simulations according to their assigned distribution. The proposed ensemble approach outperforms other agents on the game Hearthstone, including various types of MCTS. Our case study shows that uncertainty can be handled effectively using predictions of sufficient accuracy, ultimately, improving the MCTS guided decision-making process. The resulting decision-making based on such an MCTS ensemble proved to be less prone to errors by uncertainty and opens up a new class of MCTS algorithms.