Machine Learning for Game World Generation using Intrinsic Motivation as an Objective Function.

新的机器学习形式在游戏研究中越来越重要，生成性模型在内容创作方面显示出巨大的潜力。Gatys等人。[1]展示了一种使用卷积神经网络进行纹理合成的方法，该方法在所生成的纹理的保真度方面超过了现有技术的水平。Chen等人。[2]提出了一种基于强化学习的基于语义像素标签的高清图像绘制方法。Thompson等人。[3]训练卷积神经网络来模拟复杂的流体模拟，使其能够更有效地近似流体模拟。Bansal等人。[4]在竞争性的自我发挥环境中训练增强剂，导致在其他简单的环境中发展复杂的紧急行为。当这样的系统在计算上足够高效，可以实时渲染时，它们将改变视频游戏的制作和播放方式。变分、对抗性和自回归系统可以非常成功地对高维数据的统计分布进行推理。这允许模型生成具有复杂表示的内容。然而，他们不太擅长生成新奇的内容。[5]提出一个强化学习代理，其动机不是外部奖励，而是内在动机。该代理由压缩进度驱动。它不断地对世界进行采样，试图压缩它，推断数据中的模式和规律，但它的动机是寻找新的数据，“只要使它变得简单的算法规则尚未被仍在学习更好地压缩数据的适应性观察者完全吸收”，他们将其定义为‘依赖时间的主观兴趣’。我将开发由内在动机驱动的新型机器学习系统(例如依赖时间的主观兴趣[5])，优化后的系统将显示复杂的涌现行为。定义突发行为的结构和复杂性本质上是主观的[6]。拥有这种新颖性的主体--相对于底层系统中的其他结构--以及寻找它的内在动机(以目标函数的形式)，应该能够检测或鼓励产生更新结果的复杂系统的开发。研究问题-内在动机能否被用来开发高效的过程性内容生成系统，这些系统可以部署在现场游戏环境中进行纹理、模型和世界生成？内在动机能否被用来开发以复杂、不可预测的方式行为的主体行为策略或物理模拟？-哪种机器学习技术最适合实施这种方法？(强化学习、变分推理、可微神经计算机)--这项技术能否与现有的生成系统一起增强，以鼓励采样更多有趣的内容？研究计划I将从设计系统开始，该系统使用目标函数来创建由压缩过程驱动的代理。然后，这些将应用于程序内容生成和复杂代理行为的问题。在这个过程的最后，我将创建一个游戏，实现并演示这些由机器学习驱动的过程性内容生成方法。“使用卷积神经网络的纹理合成。”《神经信息处理系统的进展》，第262-270页。2015.javascript:WebForm_DoPostBackWithOptions(new%20WebForm_PostBackOptions(“ctl00$oSaveBar$btnSave”，%20“”，%20True，%20“”，%20“”，%20False，%20True)[2]陈启峰和弗拉德伦·科尔顿。“使用级联优化网络的摄影图像合成。”在IEEE国际计算机视觉会议(ICCV)，2017年第1卷。[3]Tompson，Jonathan，Kristofer Schlachter，Pablo Sprech

New forms of machine learning are increasingly important in games research, and generative models demonstrate great potential for content creation. Gatys et al. [1] demonstrate a method for texture synthesis using convolutional neural networks that exceeded the state of the art in the fidelity of of textures generated. Chen et al. [2] develop a method for high definition photographic image rendering from semantic pixel labels using reinforcement learning. Thompson et al. [3] train a convolutional neural network to model complex fluid simulations, allowing them to approximate fluid simulations more efficiently. Bansal et al. [4] train reinforcement agents in competitive self-playing environments resulting in the development of complex emergent behaviours in otherwise simple environments. When such systems are computationally efficient enough to render in real time, they will transform the way video games are both produced and played.Variational, adversarial and autoregressive systems can be very successful at performing inference on the statistical distribution of high dimensional data. This allows models to generate content with sophisticated representations. However, they are not very good at novel content generation.Schmidhuber et al. [5] propose a Reinforcement Learning agent, motivated not by external reward, but by intrinsic motivation. The agent is driven by Compression Progress. It constantly samples the world, trying to compress it, inferring patterns and regularities in the data, but motivated to seek out data that is novel "as long as the algorithmic regularity that makes it simple has not yet been fully assimilated by the adaptive observer who is still learning to compress the data better", which they define as 'the time-dependent subjective Interestingness'.I will develop new kinds of machine learning systems driven by intrinsic motivation (such as time-dependent subjective interestingness [5]), optimised towards creating systems that display complex emergent behaviour. Defining the structure and complexity of emergent behaviour is inherently subjective [6]. An agent that has this subject measure of novelty - with respect to other structures in an underlying system - and intrinsic motivation to seek it out (in the form of an objective function), should be able to detect or encourage the development of complex systems that produce more novel outcomes.Research Questions- Can Intrinsic Motivation be used to develop efficient procedural content generation systems that can be deployed in live game environments for texture, model, and world generation?- Can Intrinsic Motivation be used to develop agent behaviour policies or physical simulations that behave in complex, unpredictable ways?- What kinds of machine learning techniques are best suited to implementing this approach? (Reinforcement learning, variational inference, differentiable neural computers)- Can this technique be augmented with existing generative systems to encourage sampling more interesting content?Research Plan I will begin by designing systems that use objective functions to create agents driven by compression progress. These will then be applied to problems of procedural content generation and complex agent behaviour. At the end of this process I will create a game that implements and demonstrates these methods for procedural content generation, driven by machine learning.[1] Gatys, Leon, Alexander S. Ecker, and Matthias Bethge. "Texture synthesis using convolutional neural networks." In Advances in Neural Information Processing Systems, pp. 262-270. 2015.javascript:WebForm_DoPostBackWithOptions(new%20WebForm_PostBackOptions("ctl00$oSaveBar$btnSave",%20"",%20true,%20"",%20"",%20false,%20true))[2] Chen, Qifeng, and Vladlen Koltun. "Photographic image synthesis with cascaded refinement networks." In The IEEE International Conference on Computer Vision (ICCV), vol. 1. 2017.[3] Tompson, Jonathan, Kristofer Schlachter, Pablo Sprech