Machine Learning for Game World Generation using Intrinsic Motivation as an Objective Function.

点击翻译按钮获取中文摘要

New forms of machine learning are increasingly important in games research, and generative models demonstrate great potential for content creation. Gatys et al. [1] demonstrate a method for texture synthesis using convolutional neural networks that exceeded the state of the art in the fidelity of of textures generated. Chen et al. [2] develop a method for high definition photographic image rendering from semantic pixel labels using reinforcement learning. Thompson et al. [3] train a convolutional neural network to model complex fluid simulations, allowing them to approximate fluid simulations more efficiently. Bansal et al. [4] train reinforcement agents in competitive self-playing environments resulting in the development of complex emergent behaviours in otherwise simple environments. When such systems are computationally efficient enough to render in real time, they will transform the way video games are both produced and played.Variational, adversarial and autoregressive systems can be very successful at performing inference on the statistical distribution of high dimensional data. This allows models to generate content with sophisticated representations. However, they are not very good at novel content generation.Schmidhuber et al. [5] propose a Reinforcement Learning agent, motivated not by external reward, but by intrinsic motivation. The agent is driven by Compression Progress. It constantly samples the world, trying to compress it, inferring patterns and regularities in the data, but motivated to seek out data that is novel "as long as the algorithmic regularity that makes it simple has not yet been fully assimilated by the adaptive observer who is still learning to compress the data better", which they define as 'the time-dependent subjective Interestingness'.I will develop new kinds of machine learning systems driven by intrinsic motivation (such as time-dependent subjective interestingness [5]), optimised towards creating systems that display complex emergent behaviour. Defining the structure and complexity of emergent behaviour is inherently subjective [6]. An agent that has this subject measure of novelty - with respect to other structures in an underlying system - and intrinsic motivation to seek it out (in the form of an objective function), should be able to detect or encourage the development of complex systems that produce more novel outcomes.Research Questions- Can Intrinsic Motivation be used to develop efficient procedural content generation systems that can be deployed in live game environments for texture, model, and world generation?- Can Intrinsic Motivation be used to develop agent behaviour policies or physical simulations that behave in complex, unpredictable ways?- What kinds of machine learning techniques are best suited to implementing this approach? (Reinforcement learning, variational inference, differentiable neural computers)- Can this technique be augmented with existing generative systems to encourage sampling more interesting content?Research Plan I will begin by designing systems that use objective functions to create agents driven by compression progress. These will then be applied to problems of procedural content generation and complex agent behaviour. At the end of this process I will create a game that implements and demonstrates these methods for procedural content generation, driven by machine learning.[1] Gatys, Leon, Alexander S. Ecker, and Matthias Bethge. "Texture synthesis using convolutional neural networks." In Advances in Neural Information Processing Systems, pp. 262-270. 2015.javascript:WebForm_DoPostBackWithOptions(new%20WebForm_PostBackOptions("ctl00$oSaveBar$btnSave",%20"",%20true,%20"",%20"",%20false,%20true))[2] Chen, Qifeng, and Vladlen Koltun. "Photographic image synthesis with cascaded refinement networks." In The IEEE International Conference on Computer Vision (ICCV), vol. 1. 2017.[3] Tompson, Jonathan, Kristofer Schlachter, Pablo Sprech