Probabilistic Models for Automatic Equivariant Feature Discovery

点击翻译按钮获取中文摘要

The performance of machine learning methods is often heavily dependent on how the input data is represented, making the choice of data representation a key aspect of many applications. The field of representation learning seeks to answer questions surrounding how we can best learn meaningful representations of data. Recent interest in applying deep models to representation learning has been fueled by successes in speech recognition, categorization of object in images and language tasks such as paraphrase detection. One of the challenges of representation learning that distinguishes it from other machine learning tasks such as classification is the difficulty in establishing a clear objective for training. In this proposal, we advance a research program geared toward making progress both in our understanding of the principles behind effective representation learning and in our ability to learn representations using deep probabilistic models. Deep probabilistic models are well suited to this task as they naturally handle various forms of data corruption which are commonly seen in practice. We will explore our hypothesis that a good representation is one that separates information pertaining to different attributes of the data into different parts of the representation. For example, in images, we would like to learn to separate attributes such as object form and location. We also aim to scale-up the application of deep probabilistic models to very high dimensional data such as high resolution images and video. The primary impact of the research program will be to significantly improve the performance and range of applicability of deep probabilistic models for representation learning. While our work will focus primarily on applications with image and video data, the developed methods should be readily transferable to other application domains.