CAREER: Harnessing the Continuum for Big Data: Partial Differential Equations, Calculus of Variations, and Machine Learning

机器学习在日常生活中有着广泛的应用，如自动驾驶汽车、医学图像分析和语音识别。近年来，机器学习算法在复制甚至超过人类在其中许多任务上的表现方面取得了巨大的进步。然而，对于大规模机器学习何时能很好地发现数据中的有趣模式和结构，以及何时会失败和过度匹配，我们仍然缺乏理论上的理解。这些问题最近在对深度神经网络的敌意黑客攻击中表现出来，这在数据隐私和安全至高无上的敏感应用程序中构成了风险。这个项目的目标是利用偏微分方程组和变分理论来研究机器学习和数据科学中的基本问题，并开发基于强大理论原理的新的、更高效的算法。该项目将推进数据科学教育，为高中生开发基于图形学习的年度暑期学校，指导明尼苏达大学天才青年数学项目的高中生，开发基于图形学习当前研究的研究生课程，并支持女性和未被充分代表的群体参与该项目的所有方面。该项目的总体目标是使用离散机器学习问题的偏微分方程(PDE)连续统限制来分析现有算法，开发具有更好性能保证的新算法，并在机器学习和PDE之间建立新的联系。该项目的一个主要焦点是基于图的学习，其中图上的离散学习算法可以被解释为连续体偏微分方程组的离散化，并且这些偏微分方程组的性质(例如正则性和适定性)传达了关于学习问题的信息，并且可以产生基于强大理论原理的新算法。具体地说，该项目将(1)开发和研究一种新的算法，称为Poisson学习，用于极低标签率下的基于图的半监督学习，(2)在随机梯度下降(SGD)和非线性偏微分方程组的粘性解之间建立新的联系，表明SGD具有在不适定问题中选择好的最小化的能力，(3)利用经典椭圆正则性理论的思想来证明随机几何图上基于图的学习问题的解的内部Lipschitz正则性，(4)证明基于图的学习在低标签率下的泛化界，包括主动学习环境，以及(5)开发关于有向图上学习问题的一致性理论，例如排名算法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning has broad applications in everyday life, such as self-driving cars, medical image analysis, and speech recognition. Recent years have seen tremendous advances in the ability of machine learning algorithms to replicate and even exceed human performance on many of these tasks. However, we still lack a theoretical understanding of when large scale machine learning will work well to discover interesting patterns and structure in data, and when it will fail to do so and overfit. These problems have recently manifested in adversarial hacking of deep neural networks, which poses risks in sensitive applications where data privacy and security are paramount. The objective of this project is to use the theory of partial differential equations and the calculus of variations to study foundational problems in machine learning and data science, and develop new, more efficient, algorithms founded on strong theoretical principles. The project will advance data science education by developing an annual summer school for high school students on graph-based learning, mentoring high-school students from the University of Minnesota Talented Youth Mathematics Program, developing graduate courses on current research in graph-based learning, and supporting the participation of women and underrepresented groups in all aspects of the project.The overall goal of the project is to use partial differential equation (PDE) continuum limits for discrete machine learning problems to analyze existing algorithms, develop new algorithms with better performance guarantees, and make new connections between machine learning and PDEs. A major focus of the project is graph-based learning, where discrete learning algorithms on graphs can be interpreted as discretizations of continuum PDEs, and properties of those PDEs (e.g., regularity and well-posedness) convey information about the learning problem and can lead to new algorithms founded on strong theoretical principles. In particular, the project will (1) develop and study a new algorithm, called Poisson learning, for graph-based semi-supervised learning at very low labeling rates, (2) draw new connections between stochastic gradient descent (SGD) and viscosity solutions of nonlinear PDEs, showing that SGD has the ability to select good minimizers in ill-posed problems, (3) use ideas from classical elliptic regularity theory to prove interior Lipschitz regularity for solutions of graph-based learning problems on random geometric graphs, (4) prove generalization bounds for graph-based learning in low label-rate regimes, including the active learning setting, and (5) develop a consistency theory for learning problems on directed graphs, such as ranking algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.