Probabilistic Models for Automatic Equivariant Feature Discovery

机器学习方法的性能通常在很大程度上取决于输入数据的表示方式，这使得数据表示的选择成为许多应用程序的关键方面。表征学习领域试图回答有关我们如何最好地学习有意义的数据表征的问题。最近，在语音识别、图像中对象分类和语言任务(如释义检测)方面的成功推动了将深度模型应用于表征学习的兴趣。表征学习区别于分类等其他机器学习任务的挑战之一是难以建立明确的训练目标。在这个提案中，我们提出了一个研究计划，旨在促进我们对有效表征学习背后的原理的理解，以及我们使用深度概率模型学习表征的能力。深度概率模型非常适合于这项任务，因为它们可以自然地处理实践中常见的各种形式的数据损坏。我们将探索我们的假设，即一个好的表示是将与数据的不同属性有关的信息分离到表示的不同部分。例如，在图像中，我们想要学习区分对象形式和位置等属性。我们还计划将深度概率模型的应用扩大到超高维数据，如高分辨率图像和视频。该研究计划的主要影响将是显著提高深度概率模型的性能和适用范围，以进行表征学习。虽然我们的工作主要集中在图像和视频数据的应用上，但所开发的方法应该可以很容易地移植到其他应用领域。

The performance of machine learning methods is often heavily dependent on how the input data is represented, making the choice of data representation a key aspect of many applications. The field of representation learning seeks to answer questions surrounding how we can best learn meaningful representations of data. Recent interest in applying deep models to representation learning has been fueled by successes in speech recognition, categorization of object in images and language tasks such as paraphrase detection. One of the challenges of representation learning that distinguishes it from other machine learning tasks such as classification is the difficulty in establishing a clear objective for training. In this proposal, we advance a research program geared toward making progress both in our understanding of the principles behind effective representation learning and in our ability to learn representations using deep probabilistic models. Deep probabilistic models are well suited to this task as they naturally handle various forms of data corruption which are commonly seen in practice. We will explore our hypothesis that a good representation is one that separates information pertaining to different attributes of the data into different parts of the representation. For example, in images, we would like to learn to separate attributes such as object form and location. We also aim to scale-up the application of deep probabilistic models to very high dimensional data such as high resolution images and video. The primary impact of the research program will be to significantly improve the performance and range of applicability of deep probabilistic models for representation learning. While our work will focus primarily on applications with image and video data, the developed methods should be readily transferable to other application domains.