Generative Modelling and Representation Learning

点击翻译按钮获取中文摘要

Raw data is abundant in the modern world. Creating models that can make sense of this large flow of information would be very helpful for many tasks. Unfortunately, most data is not neatly packaged in a format that traditional machine learning methods can use. It is desirable to have methods that can extract useful information from data in any format. A learning paradigm that meets this criteria is generative modelling. Generative models look at example data from a certain source and learn to synthesize new fake data that looks like the original real data. Synthesizing fake examples may not be useful for many applications in and of itself. However, inherent in the ability to create realistic examples is a deep understand of the structure and form of the data. Therefore, within these generative models there must exist 'representations' of the data that summarize pertinent aspects of the data such as its structure and form. These representations are useful for both humans and further models. The generative models can be coaxed into producing representations that are interpretable to humans providing us with automatic and extensive summaries of large amounts of data that go beyond simple metrics such as the mean and variance. Parallel to this, further models can be trained directly on the representations of the data instead of on the raw data itself. Since the representations contain condensed information about the data learnt by the original generative model, it is usually the case that models trained on representations require much less overall data to achieve the same level of performance as a model trained on the raw data directly.This project aims to improve upon existing generative modelling techniques as well as formulate new ways to extract representations from learnt generative models. New methods for generative modelling are regularly proposed in the research community but our understanding of exactly what is being learnt from the data and how to extract this information often lags behind. The project will deal with very novel methods for generative modelling and aims to bring our understanding of their inner workings up to speed. This will be achieved through a combination of empirical investigation into state of the art models as well as theoretical work to characterise their behaviours.This project falls within the EPSRC Artifical Intelligence and Robotics research area.Collaboration is currently planned within the Oxford University Department of Statistics internally.