Collaborative Research: Geometric Analysis and Computation for Generative Models

无监督学习和生成模型的研究涉及揭示数据中的结构和关系，目的是能够生成新的，尚未见过的数据集示例。生成模型从有限样本中学习数据集的分布，并提供近似密度的有效采样器，而不是依赖标签进行监督。这些模型对于以无监督的方式分析大容量、高维数据是一个强大的工具。虽然生成模型是机器学习中一个活跃的研究课题，但这些模型的许多理论和计算问题仍不清楚。该合作研究项目将从几何角度研究生成模型，重点关注性能保证和有效实现。有效地创建保证与现有数据相似的新数据点的能力在各种应用中具有重要意义，包括医疗数据分析和隐私、生物信息学、图像和音频信号建模，以及难以为监督算法收集标记数据的一般高维数据分析。这个研究项目的思想和方法围绕着过去十年来在多元学习领域发展起来的技术。这些数学工具，特别是局部邻域保持图、基于内在维数的近似分析和基于局部亲和力的全局坐标系统的构建，在生成模型的研究中具有天然的应用。该项目由该领域出现的四个基本问题组成：(a)生成网络能够有效学习的分布类型是什么，分布的内在维度如何影响收敛？(b)如何为流形学习中出现的仅取决于数据的内在几何形状的降维表示创建非参数生成模型？(c)如何在高维分布之间确定有效计算的度量，以用于评估各种生成模型的有效性？(d)如何使用这些度量来检查生成模型在接受训练时通过参数空间的各种路径，以及起始点的哪一组提供最佳生成器？该项目将重点关注这些问题的数学和计算方面，旨在解决有关这些工具的基本问题，这些工具广泛用于科学和工业中的各种数据分析和信号处理应用。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Research in unsupervised learning and generative models is concerned with uncovering structure and relationships in data with the intent of being able to generate new, as yet unseen, examples of the data set. Generative models learn the distribution of a data set from finite samples and provide an efficient sampler of the approximated density, rather than relying on labels for supervision. These models are a powerful tool for analyzing large volume, high-dimensional data in an unsupervised way. While generative models are an active research topic in machine learning, many theoretical and computational questions for such models remain unclear. This collaborative research project will study generative models from a geometric perspective, focusing on both performance guarantees and efficient implementations. The ability to efficiently create new data points that are guaranteed to be similar to the existing data has important implications in a variety of applications, including medical data analysis and privacy, bioinformatics, modeling of image and audio signals, and general high-dimensional data analysis in which it is difficult to collect labeled data for supervised algorithms.The ideas and approaches in this research project center around the techniques that have evolved in the manifold learning field over the past decade. These mathematical tools, in particular local neighborhood preserving maps, approximation analysis in terms of intrinsic dimensionality, and construction of global coordinate systems based upon local affinity, have natural applications in the study of generative models. The project is comprised of four fundamental questions that arise in the field: (a) What are the types of distributions that generative networks are capable of learning efficiently, and how does the intrinsic dimensionality of the distribution affect convergence? (b) How can non-parametric generative models be created for dimension-reduced representations that arise in manifold learning, and which only depend on the intrinsic geometry of the data? (c) How can efficiently-computed metrics be defined between high-dimensional distributions for use in assessing the validity of various generative models? (d) How can these metrics be used to examine the various paths generative models take through the parameter space while being trained, and what clusters of starting points give optimal generators? The project will focus on both mathematical and computational aspects of these problems, aiming at resolving fundamental questions about these tools that are widely used in various data analysis and signal processing applications in science and industry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.