Improving AI Performance in Complex Real-Time Scenarios with Hierarchical Reinforcement Learning and Counterfactual Regret Minimization

最近游戏人工智能(AI)的里程碑带来了围棋、国际象棋、扑克、Dota2和星际争霸II等游戏的专家解决方案。特别是最后一个标题强调了AI在复杂的实时场景中的适用性，但只能通过大规模神经网络(NN)架构。分层强化学习为主体的决策过程提供了管理结构，而反事实后悔最小化通过事后学习来评估行动决策。两者的结合可能会减少网络规模，代理需要处理与星际争霸II同样困难的环境，甚至更复杂的环境，如战争游戏。一旦成功，本研究开发的方法可以在实时策略(RTS)游戏环境中达到最先进的性能，不仅可以用来增强玩家在各种RTS游戏中的体验，还可以推动现实世界决策场景中的人工智能就业能力。

Recent milestones in Game Artificial Intelligence (AI) brought forth expert solutions for playing the games of Go, Chess, Poker, Dota2, and StarCraft II. Especially the last title highlighted the applicability of AI in complex real-time scenarios, yet only through massive Neural Network (NN) architectures. Hierarchical Reinforcement Learning provides managerial structure to an agent's decision-making process, while Counterfactual Regret Minimization evaluates action-decision making by learning from hindsight. Combining both may reduce the network size, that agents require to handle environments equally difficult to StarCraft II, or even more complex ones, such as Wargames. Upon success, the methods developed in this study could reach state-of-the-art performance in Real-Time Strategy (RTS) games environments, which could not only be used to enhance the experience of players in diverse RTS games but also push forward AI employability in real-world decision-making scenarios.