Collaborative Research: Geometric Analysis and Computation for Generative Models

无监督学习和生成模型的研究关注于揭示数据中的结构和关系，目的是能够生成数据集的新的、尚未看到的示例。生成模型从有限样本中学习数据集的分布，并提供近似密度的有效采样器，而不是依赖标签进行监督。这些模型是以非监督方式分析大容量、高维数据的强大工具。虽然产生式模型是机器学习中的一个活跃的研究主题，但对于这类模型的许多理论和计算问题仍然不清楚。该合作研究项目将从几何角度研究产生式模型，重点关注性能保证和高效实现。高效地创建与现有数据相似的新数据点的能力在各种应用中具有重要意义，包括医疗数据分析和隐私、生物信息学、图像和音频信号的建模，以及难以为监督算法收集标记数据的一般高维数据分析。本研究项目的思想和方法围绕着过去十年来在多种学习领域中发展起来的技术。这些数学工具，特别是局部邻域保持映射、基于本征维度的逼近分析和基于局部亲和力的全局坐标系的构造，在生成模型的研究中有着天然的应用。该项目由该领域出现的四个基本问题组成：(A)生成性网络能够有效学习的分布类型是什么，以及分布的内在维度如何影响收敛？(B)如何为流形学习中出现的降维表示创建非参数生成模型，并且这些表示只依赖于数据的固有几何形状？(C)如何在高维分布之间定义有效计算的度量，以用于评估各种生成模型的有效性？(D)如何使用这些度量来检查生成模型在被训练时通过参数空间的各种路径，以及哪些起点集群提供了最优生成器？该项目将专注于这些问题的数学和计算方面，旨在解决这些工具的基本问题，这些工具广泛应用于科学和工业的各种数据分析和信号处理应用程序。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Research in unsupervised learning and generative models is concerned with uncovering structure and relationships in data with the intent of being able to generate new, as yet unseen, examples of the data set. Generative models learn the distribution of a data set from finite samples and provide an efficient sampler of the approximated density, rather than relying on labels for supervision. These models are a powerful tool for analyzing large volume, high-dimensional data in an unsupervised way. While generative models are an active research topic in machine learning, many theoretical and computational questions for such models remain unclear. This collaborative research project will study generative models from a geometric perspective, focusing on both performance guarantees and efficient implementations. The ability to efficiently create new data points that are guaranteed to be similar to the existing data has important implications in a variety of applications, including medical data analysis and privacy, bioinformatics, modeling of image and audio signals, and general high-dimensional data analysis in which it is difficult to collect labeled data for supervised algorithms.The ideas and approaches in this research project center around the techniques that have evolved in the manifold learning field over the past decade. These mathematical tools, in particular local neighborhood preserving maps, approximation analysis in terms of intrinsic dimensionality, and construction of global coordinate systems based upon local affinity, have natural applications in the study of generative models. The project is comprised of four fundamental questions that arise in the field: (a) What are the types of distributions that generative networks are capable of learning efficiently, and how does the intrinsic dimensionality of the distribution affect convergence? (b) How can non-parametric generative models be created for dimension-reduced representations that arise in manifold learning, and which only depend on the intrinsic geometry of the data? (c) How can efficiently-computed metrics be defined between high-dimensional distributions for use in assessing the validity of various generative models? (d) How can these metrics be used to examine the various paths generative models take through the parameter space while being trained, and what clusters of starting points give optimal generators? The project will focus on both mathematical and computational aspects of these problems, aiming at resolving fundamental questions about these tools that are widely used in various data analysis and signal processing applications in science and industry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.