Collaborative Research: Geometric Analysis and Computation for Generative Models

协作研究：生成模型的几何分析和计算

• 批准号: 1819222
• 负责人: Alexander Cloninger
• 金额: $ 19.59万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1819222&HistoricalAwards=false
• 关键词: Collaborative; Research; Geometric; Analysis; Computation

Research in unsupervised learning and generative models is concerned with uncovering structure and relationships in data with the intent of being able to generate new, as yet unseen, examples of the data set. Generative models learn the distribution of a data set from finite samples and provide an efficient sampler of the approximated density, rather than relying on labels for supervision. These models are a powerful tool for analyzing large volume, high-dimensional data in an unsupervised way. While generative models are an active research topic in machine learning, many theoretical and computational questions for such models remain unclear. This collaborative research project will study generative models from a geometric perspective, focusing on both performance guarantees and efficient implementations. The ability to efficiently create new data points that are guaranteed to be similar to the existing data has important implications in a variety of applications, including medical data analysis and privacy, bioinformatics, modeling of image and audio signals, and general high-dimensional data analysis in which it is difficult to collect labeled data for supervised algorithms.The ideas and approaches in this research project center around the techniques that have evolved in the manifold learning field over the past decade. These mathematical tools, in particular local neighborhood preserving maps, approximation analysis in terms of intrinsic dimensionality, and construction of global coordinate systems based upon local affinity, have natural applications in the study of generative models. The project is comprised of four fundamental questions that arise in the field: (a) What are the types of distributions that generative networks are capable of learning efficiently, and how does the intrinsic dimensionality of the distribution affect convergence? (b) How can non-parametric generative models be created for dimension-reduced representations that arise in manifold learning, and which only depend on the intrinsic geometry of the data? (c) How can efficiently-computed metrics be defined between high-dimensional distributions for use in assessing the validity of various generative models? (d) How can these metrics be used to examine the various paths generative models take through the parameter space while being trained, and what clusters of starting points give optimal generators? The project will focus on both mathematical and computational aspects of these problems, aiming at resolving fundamental questions about these tools that are widely used in various data analysis and signal processing applications in science and industry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无监督学习和生成模型的研究关注的是发现数据中的结构和关系，目的是能够生成新的，尚未见过的数据集示例。生成模型从有限样本中学习数据集的分布，并提供近似密度的有效采样器，而不是依赖于标签进行监督。这些模型是以无监督的方式分析大容量，高维数据的强大工具。虽然生成模型是机器学习中一个活跃的研究课题，但此类模型的许多理论和计算问题仍然不清楚。这个合作研究项目将从几何角度研究生成模型，重点是性能保证和高效实现。有效地创建保证与现有数据相似的新数据点的能力在各种应用中具有重要意义，包括医疗数据分析和隐私、生物信息学、图像和音频信号的建模，一般高-多维数据分析是一种很难为监督算法收集标记数据的数据分析方法，本研究项目的思想和方法围绕着在过去的十年里，在多方面的学习领域中不断发展。这些数学工具，特别是局部邻域保持地图，近似分析的内在维度，和建设的全球坐标系的基础上，当地的亲和力，有自然的应用在生成模型的研究。该项目由该领域出现的四个基本问题组成：（a）生成网络能够有效学习的分布类型是什么，分布的内在维度如何影响收敛？(b)如何为流形学习中出现的降维表示创建非参数生成模型，并且这些降维表示仅取决于数据的内在几何形状？(c)如何在高维分布之间定义有效计算的度量，以用于评估各种生成模型的有效性？(d)如何使用这些指标来检查生成模型在训练时通过参数空间的各种路径，以及哪些起始点集群可以提供最佳生成器？该项目将集中在这些问题的数学和计算方面，旨在解决这些工具的基本问题，这些工具广泛用于科学和工业中的各种数据分析和信号处理应用。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Natural Graph Wavelet Packet Dictionaries

自然图小波包字典

• DOI: 10.1007/s00041-021-09832-3
• 发表时间: 2021
• 期刊: Journal of Fourier Analysis and Applications
• 影响因子: 1.2
• 作者: Cloninger, Alexander;Li, Haotian;Saito, Naoki
• 通讯作者: Saito, Naoki

Classification Logit Two-Sample Testing by Neural Networks for Differentiating Near Manifold Densities

• DOI: 10.1109/tit.2022.3175691
• 发表时间: 2019-09
• 期刊: IEEE Transactions on Information Theory
• 影响因子: 2.5
• 作者: Xiuyuan Cheng;A. Cloninger

StreaMRAK a streaming multi-resolution adaptive kernel algorithm

• DOI: 10.1016/j.amc.2022.127112
• 发表时间: 2021-08
• 期刊: Appl. Math. Comput.
• 作者: Andreas Oslandsbotn;Ž. Kereta;Valeriya Naumova;Y. Freund;A. Cloninger

Nonclosedness of sets of neural networks in Sobolev spaces

• DOI: 10.1016/j.neunet.2021.01.007
• 发表时间: 2021-01
• 期刊: Neural networks : the official journal of the International Neural Network Society
• 作者: Scott Mahan;E. King;A. Cloninger

PT-MMD: A Novel Statistical Framework for the Evaluation of Generative Systems

• DOI: 10.1109/ieeeconf44664.2019.9048757
• 发表时间: 2019-10
• 期刊: 2019 53rd Asilomar Conference on Signals, Systems, and Computers
• 作者: Alexander Potapov;Ian Colbert;K. Kreutz-Delgado;A. Cloninger;Srinjoy Das