AF: Medium: Collaborative Research: Theoretical Foundations of Deep Generative Models and High-Dimensional Distributions

点击翻译按钮获取中文摘要

Current technology is driving our ability to collect, store and process data at an unprecedented scale. Ranging from image, audio and video to social-network, medical and biological datasets, modern applications require us to model and reason about complex data over extremely large domains. It is well-known, however, that this cannot be done in a rigorous manner unless simplifying assumptions can be made about how the data of interest are generated. Accordingly, a long line of investigation in Probability Theory, Statistical Physics, Information Theory and Machine Learning has been preoccupied with developing mathematical and algorithmic frameworks that allow for succinct representation and inference of high-dimensional distributions with simplifying structure. This project will go beyond the standard frameworks in these fields to advance the theoretical foundations of a research frontier that has recently emerged as a promising approach towards a more accurate modeling of high-dimensional data. In particular, this project will study the theoretical foundations of learning, testing and statistical inference of high-dimensional data that are generated by deep neural network-based generative models, developing mathematically rigorous quality guarantees, which is a big desideratum in the field of deep learning. On the practical front, this work has the potential to significantly improve the performance of image-reconstruction algorithms compared to state-of-the-art, and therefore to have significant impact on various applications of image reconstruction such as rapid magnetic resonance imaging (MRI).Since the introduction of deep neural network-based generative models, there have been numerous approaches for how to architect them, how to train them using samples from a distribution of interest, and how to use them for downstream inference tasks; these have delivered impressive practical results. On the other hand, there has also been a lot of debate around the quality of deep generative models that are trained via current techniques, and it has been recognized that there are significant challenges in optimizing, evaluating and scaling the dimensionality of deep generative models, as well as in using them for data recovery. This project develops three research thrusts targeting these challenges, namely: (i) developing better algorithms for training deep generative models, and for using these models as "regularizers" in signal-processing applications; (ii) developing statistical techniques for evaluating the quality of a deep generative model against the distribution whose samples it was trained on; (iii) proposing architectures and algorithms for scaling up the dimensionality of deep generating models while providing statistical accuracy guarantees. This work will rely on techniques from non-convex and combinatorial optimization, signal processing, game theory, high-dimensional statistics, and statistical physics, and build connections between these fields and deep learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.