CAREER: Information-Theoretic and Statistical Foundations of Generative Models

生成式机器学习模型提供了对数据的统计理解，并在现代机器学习的各个应用领域（包括视觉、语音、自然语言和计算生物学等）的成功中发挥着重要作用。基于深度学习的成功，现代生成模型的最新进展为各种学习方法的革命带来了巨大的希望。尽管取得了这些进展，但对这些模型的一些基本方面的理解仍处于起步阶段，这些方面是表征其性能保证所必需的。该项目旨在通过利用信息论、统计学和优化的工具和概念，阐明现代生成模型的统计和计算特性。该项目还包括一个综合计划，将研究成果整合到一个包容、多样化和跨学科的高中、本科和研究生教育计划中。该研究计划的总体目标是对现代生成模型（如生成对抗网络（gan）和变分自动编码器（VAEs））的统计和计算方面进行全面和基本的理解。本项目旨在对高维分布的生成模型的适当表述，表征这些模型的统计限制，以及开发有效的计算方法来解决训练过程中涉及的优化问题取得关键进展。这个跨学科的项目扩大了信息理论和机器学习之间相互作用的先验知识的范围，并在数据科学的理论、算法和应用之间建立了一个紧密联系的循环。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Generative machine learning models provide a statistical understanding of data and play an important role in the success of modern machine learning in various application domains including vision, speech, natural languages, and computational biology, among others. Building on the success of deep learning, recent advances in modern generative models hold great promise in revolutionizing various learning methods. Despite this progress, the understanding of some fundamental aspects of these models, required for characterizing their performance guarantees, is still in its infancy. This project aims to elucidate statistical and computational properties of modern generative models by leveraging tools and concepts from information theory, statistics and optimization. This project also includes a comprehensive plan to integrate the research results into an inclusive, diverse and cross-disciplinary educational program at the high school, undergraduate and graduate levels. The overall goal of the research program is to develop a comprehensive and fundamental understanding of the intertwined statistical and computational aspects of modern generative models such as Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs). This project aims to make critical advances in proper formulations of generative models for high dimensional distributions, characterizing statistical limits of these models, and developing efficient computational approaches for solving optimization problems involved during their training. This cross-disciplinary project broadens the scope of the prior knowledge on the interplay between information theory and machine learning and creates a tightly connected loop between theory, algorithms and applications in data science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.