Variational Methods for the Analysis of Unsupervised Learning Algorithms

无监督学习可以被描述为从未标记的观测中发现结构的任务。这是生物学、材料科学、经济学和天文学等科学领域的一项基本任务，在这些领域中，典型的情况下，研究人员拥有大量未标记的数据，但只有少数(或没有)专家标记的数据点，因为这些数据点通常非常昂贵。本项目致力于无监督学习的数据科学算法的数学分析。总的来说，调查的目的是通过提供严格的数学理论支持实践中使用的方法，以及提出原则性的新方法，为数据科学和人工智能的数学基础的发展做出贡献。研究者不仅会研究单个算法，而且会发现它们之间的共同结构，并突出它们之间的不明确的统一性。在文献中发现的许多非监督学习方法中，基于图的方法占据了突出的地位，它们将是本次调查的重点。这些算法依赖于能够访问相似图，从而赋予数据集几何结构。不同的图表捕捉给定数据集的不同特征，因此对不同的任务很有用。该项目将集中在以下具体方向：1)提出和分析新的无监督学习算法，该算法的灵感来自于度量空间上的进化的变分方法，特别是离散和半离散结构上的概率度量空间。在这个方向上，主要目标是为设计新的学习方法引入一种数学形式主义，并利用这种形式主义在看似无关的统计程序之间建立联系，比如那些受到模式寻找技术启发的统计程序与基于谱方法的统计程序。考虑到的一些应用超越了无监督学习，还包括对深度神经网络结构的系统测试。第二个方向涉及基于图的无监督学习算法的统计特性的研究，在这种情况下，捕捉数据集的几何的图是非常稀疏的(低于标准连通性)，或者数据生成分布是退化的并且违反了文献中发现的标准几何假设。在这个方向上，目标是确定稀疏性的极限，允许有意义的学习，以及为建立在数据集上的图形上的非凸优化问题的解决方案开发定量的统计误差分析，用于连接到基本真实几何模型的图形。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Unsupervised learning can be described as the task of finding structure from unlabeled observations. This is a fundamental task in scientific fields like biology, material science, economics, and astronomy, where in a typical scenario researchers possess a large number of unlabeled data, but only a few (or none) expert labeled data points, as these are typically very expensive to obtain. This project focuses on the mathematical analysis of data science algorithms for unsupervised learning. Broadly speaking, the aim of the investigation is to contribute to the development of mathematical foundations for data science and artificial intelligence by providing rigorous mathematical theory supporting methodologies used in practice, as well as by proposing principled novel ones. The investigator will not only study individual algorithms, but find common structure between them and highlight their otherwise unclear unity.Among the many methodologies for unsupervised learning found in the literature, graph-based methods occupy a prominent role, and they will be the focus of this investigation. These are algorithms that rely on having access to similarity graphs, which endow data sets with a geometric structure. Different graphs capture different features of a given data set and hence are useful for different tasks. The project will focus on the following concrete directions: 1) the proposal and analysis of new algorithms for unsupervised learning inspired by variational methods for evolution on metric spaces and in particular on the space of probability measures over discrete and semi-discrete structures. In this direction the main goal is to introduce a mathematical formalism for the design of new learning methodologies, and to use this formalism in order to build connections between seemingly unrelated statistical procedures like those inspired by mode-seeking techniques as the ones based on spectral methods. Some of the applications in mind go beyond unsupervised learning, and also include the systematic testing of deep neural network architectures. The second direction concerns the study of statistical properties of graph-based unsupervised learning algorithms in settings where graphs capturing the geometry of a data set are very sparse (below standard connectivity), or where data generating distributions are degenerate and violate standard geometric assumptions found in the literature. In this direction, the goal is to identify the limits of sparsity that allow for meaningful learning, as well as to develop quantitative statistical error analysis for solutions to non-convex optimization problems on graphs built on data sets, for graphs that are connected to a ground truth geometric model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.