CAREER: Information-Theoretic and Statistical Foundations of Generative Models

生成式机器学习模型提供了对数据的统计理解，并在现代机器学习在各种应用领域（包括视觉、语音、自然语言和计算生物学等）的成功中发挥着重要作用。在深度学习成功的基础上，现代生成模型的最新进展在革新各种学习方法方面具有巨大的潜力。尽管取得了这一进展，这些模型的一些基本方面的理解，需要表征其性能保证，仍然处于起步阶段。该项目旨在通过利用信息论，统计学和优化的工具和概念来阐明现代生成模型的统计和计算特性。该项目还包括一个全面的计划，将研究成果纳入高中，本科和研究生阶段的包容性，多样性和跨学科的教育计划。该研究计划的总体目标是对生成对抗网络（GAN）和变分自动编码器（VAE）等现代生成模型的相互交织的统计和计算方面有一个全面而基本的了解。该项目旨在为高维分布的生成模型的适当配方，表征这些模型的统计限制，并开发有效的计算方法来解决其训练过程中涉及的优化问题。这个跨学科的项目拓宽了信息理论和机器学习之间相互作用的现有知识的范围，并在数据科学的理论、算法和应用之间建立了一个紧密联系的循环。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Generative machine learning models provide a statistical understanding of data and play an important role in the success of modern machine learning in various application domains including vision, speech, natural languages, and computational biology, among others. Building on the success of deep learning, recent advances in modern generative models hold great promise in revolutionizing various learning methods. Despite this progress, the understanding of some fundamental aspects of these models, required for characterizing their performance guarantees, is still in its infancy. This project aims to elucidate statistical and computational properties of modern generative models by leveraging tools and concepts from information theory, statistics and optimization. This project also includes a comprehensive plan to integrate the research results into an inclusive, diverse and cross-disciplinary educational program at the high school, undergraduate and graduate levels. The overall goal of the research program is to develop a comprehensive and fundamental understanding of the intertwined statistical and computational aspects of modern generative models such as Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs). This project aims to make critical advances in proper formulations of generative models for high dimensional distributions, characterizing statistical limits of these models, and developing efficient computational approaches for solving optimization problems involved during their training. This cross-disciplinary project broadens the scope of the prior knowledge on the interplay between information theory and machine learning and creates a tightly connected loop between theory, algorithms and applications in data science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.