Abstract Forward Models for Modern Games

游戏产业是世界上增长最快的产业之一，年收入预计将从2018年的1380亿美元增加到2021年的1800亿美元。英国游戏产业是全球领先的产业，为财富创造和出口做出了巨大贡献，并呈现出明显的增长趋势：英国2261家公司中有62%是在过去8年里成立的。该行业拥有12000名员工，2017年销售额达43亿英镑，是欧洲第二大、世界第五大市场。游戏也是AI发展的优秀基准。最近最明显的例子之一就是围棋游戏中搜索方法的进展。围棋是一种有上千年历史的棋盘游戏，规则简单，策略复杂，自该领域诞生以来，人类一直主宰着计算机人工智能。蒙特卡罗树搜索（MCTS）是一种人工智能技术，可以探索棋手可以采取的不同分支动作，在2016年成为创建围棋人工智能棋手的标准算法，催生了对该算法的变化和应用的大量研究。从那时起，MCTS已经在游戏内外的数千个其他作品中使用。当谷歌Deepmind的阿尔法围棋结合MCTS和深度学习（DL）掌握了这一游戏时，这一进展达到了另一个里程碑。MCTS使用前向模型（FM），这是游戏状态的一种表示，允许在应用游戏中的任何动作后向前滚动状态。这个“模拟器”也被其他统计前向规划（SFP）方法所使用，这些方法在某些领域也显示出与MCTS类似的前景，比如滚动地平线进化算法（RHEA）。然而，令人惊讶的是，尽管SFP方法很受欢迎并取得了进展，但它们几乎没有进入游戏行业。在游戏行业中，最著名的将MCTS用于对手AI的例子是Creative Assembly的《全面战争》系列，纸牌游戏中的AI Factory以及Lionhead的《Fable Legends》战术规划。考虑到游戏行业是世界和英国发展最快的行业之一，人们可能会想，为什么游戏中AI的顶级算法之一仅占该行业的0.01%。这个项目的目的是将一个FM库整合到一个现代游戏引擎中，以促进在大型、复杂的视频游戏中使用SFP技术的研究。一方面，该项目将解决集成可定制FM的技术和设计问题，可定制FM决定真实游戏状态的哪些元素构成FM的一部分，以及如何进行抽象。另一方面，该项目旨在了解SFP方法在复杂的大型商业游戏中如何在这些条件下执行，并研究如何改进这些方法。由此产生的框架将允许在广泛的游戏中测试这些方法，特别强调为行业和研究人员提出游戏AI竞赛。项目研究成果的传播将通过开源库、框架、文档和科学论文得到保证。这个项目自然地建立在PI最近在GVGAI上的工作上（他是比赛，赛道和团队的主要开发者，组织者和协调员- www.gvgai.net），它提出了通用游戏AI研究及其与游戏产业的相关性的一个步骤改变，使其适应现代游戏。该项目直接解决了MCTS/RHEA等成熟方法在大型复杂游戏中的适用性，以及行业对快速、可靠和先进AI技术的需求。我们强大的游戏行业合作伙伴（Microsoft Research, AI Factory, Bossa Studios， Creative Assembly和Gwaredd Mountain）将帮助引导项目进入游戏研究和行业社区的利益。游戏以外的应用也将在我们的非游戏行业合作伙伴（国防科学技术实验室）的帮助下进行探索。

The games industry is one of the fastest-growing industries in the world, with yearly revenues expected to increase from US$ 138bn in 2018 to US$ 180bn in 2021. The UK games industry is a worldwide leader that contributes significantly to wealth creation and export, with a clear growing tendency: 62% of the 2261 companies in the UK were founded in the last 8 years. Employing 12,000 people with sales valued in £4.3bn for 2017, this industry is the second largest market in Europe and the fifth in the world. Games have also been excellent benchmarks for the advancement of AI. One of the most clear and recent examples of this is the progress on search methods in the game of Go. Go is a thousand years old board game of simple rules but complex strategy, where humans had dominated computer AIs since the beginning of the field. Monte Carlo Tree Search (MCTS), an AI technique that explores the different branches of actions that both players can take, became in 2016 the standard algorithm for creating Go AI players, giving birth to substantial research on variations and applications of this algorithm. Since then, MCTS has been used in thousands of other works in and outside games. This progress reached another milestone when Google Deepmind's Alpha Go mastered this game with a combination of MCTS and Deep Learning (DL).MCTS uses a forward model (FM), which is a representation of the game state that allows to roll the state forward after applying any action in the game. This "simulator" is also used by other Statistical Forward Planning (SFP) methods that are also showing similar promise to MCTS in some domains, such as Rolling Horizon Evolutionary Algorithms (RHEA). It is however striking that despite the popularity and progress on SFP methods, they have barely reached the games industry. The most known uses of MCTS for Opponent AI in the games industry are in the Total War series by Creative Assembly, AI Factory on card games and Lionhead's tactical planning for Fable Legends. Given that the games industry is one of the fastest growing industries in the world and UK one may wonder why one of the top algorithms on AI in Games barely reaches far less than 0.01% of this industry.The aim of this project is to incorporate an FM library into a modern games engine in order to facilitate research on the use of SFP techniques in large, complex, video-games. On the one hand, the project will address the technical and design problems of integrating a customisable FM that determines which elements of the real game state form part of the FM and how abstractions can be made. On the other hand, the project will aim to understand how SFP methods perform under these conditions in complex and large commercial-like games, investigating how these can be improved. The resultant framework will allow to test these methods in a wide range of games, with a special emphasis on proposing a Game AI competition for industry and researchers. Dissemination of the project's research outcomes will be guaranteed via open source libraries, frameworks, documentation and scientific papers. This project builds naturally on the PI's recent work on GVGAI (for which he is main developer, organiser and coordinator of the competition, tracks and team - www.gvgai.net), and it proposes a step change on General Game AI research and its relevance to the games industry, adapting it to modern games. This project addresses directly the applicability of well-established methods such as MCTS/RHEA to large and complicated games and also the industry needs for fast, reliable and state of the art AI techniques. Our strong group of game industry partners (Microsoft Research, AI Factory, Bossa Studios, Creative Assembly and Gwaredd Mountain) will help steer the project into the interests of the game research and industry communities. Applications beyond games will also be explored with the help of our non-game industry partner (the Defence Science and Technology Laboratory).