Variational Methods for the Analysis of Unsupervised Learning Algorithms

无监督学习可以被描述为从未标记的观察中发现结构的任务。这是生物学、材料科学、经济学和天文学等科学领域的一项基本任务，在典型情况下，研究人员拥有大量未标记的数据，但只有少数（或没有）专家标记的数据点，因为这些数据点通常非常昂贵。该项目专注于无监督学习的数据科学算法的数学分析。从广义上讲，调查的目的是通过提供严格的数学理论支持实践中使用的方法，以及提出原则性的新颖方法，为数据科学和人工智能的数学基础的发展做出贡献。研究者不仅要研究单个算法，还要找到它们之间的共同结构，并突出它们之间原本不明确的统一性。在文献中发现的许多无监督学习方法中，基于图的方法占据了突出的地位，它们将成为本次调查的重点。这些算法依赖于访问相似性图，这些相似性图赋予数据集几何结构。不同的图捕捉给定数据集的不同特征，因此对不同的任务有用。该项目将集中在以下具体方向：1）建议和分析新的无监督学习算法，其灵感来自度量空间，特别是离散和半离散结构上的概率测度空间上的变分进化方法。在这个方向上，主要目标是引入一个数学形式主义的设计新的学习方法，并使用这种形式主义，以建立之间的联系，看似无关的统计程序，如那些启发模式搜索技术的基础上的频谱方法。考虑中的一些应用超出了无监督学习，还包括深度神经网络架构的系统测试。第二个方向涉及研究基于图的无监督学习算法的统计属性，其中捕获数据集几何形状的图非常稀疏（低于标准连接性），或者数据生成分布退化并违反标准几何假设。在文献中。在这个方向上，目标是确定允许有意义学习的稀疏性限制，以及为基于数据集构建的图上的非凸优化问题的解决方案开发定量统计误差分析，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值进行评估来支持和更广泛的影响审查标准。

Unsupervised learning can be described as the task of finding structure from unlabeled observations. This is a fundamental task in scientific fields like biology, material science, economics, and astronomy, where in a typical scenario researchers possess a large number of unlabeled data, but only a few (or none) expert labeled data points, as these are typically very expensive to obtain. This project focuses on the mathematical analysis of data science algorithms for unsupervised learning. Broadly speaking, the aim of the investigation is to contribute to the development of mathematical foundations for data science and artificial intelligence by providing rigorous mathematical theory supporting methodologies used in practice, as well as by proposing principled novel ones. The investigator will not only study individual algorithms, but find common structure between them and highlight their otherwise unclear unity.Among the many methodologies for unsupervised learning found in the literature, graph-based methods occupy a prominent role, and they will be the focus of this investigation. These are algorithms that rely on having access to similarity graphs, which endow data sets with a geometric structure. Different graphs capture different features of a given data set and hence are useful for different tasks. The project will focus on the following concrete directions: 1) the proposal and analysis of new algorithms for unsupervised learning inspired by variational methods for evolution on metric spaces and in particular on the space of probability measures over discrete and semi-discrete structures. In this direction the main goal is to introduce a mathematical formalism for the design of new learning methodologies, and to use this formalism in order to build connections between seemingly unrelated statistical procedures like those inspired by mode-seeking techniques as the ones based on spectral methods. Some of the applications in mind go beyond unsupervised learning, and also include the systematic testing of deep neural network architectures. The second direction concerns the study of statistical properties of graph-based unsupervised learning algorithms in settings where graphs capturing the geometry of a data set are very sparse (below standard connectivity), or where data generating distributions are degenerate and violate standard geometric assumptions found in the literature. In this direction, the goal is to identify the limits of sparsity that allow for meaningful learning, as well as to develop quantitative statistical error analysis for solutions to non-convex optimization problems on graphs built on data sets, for graphs that are connected to a ground truth geometric model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.