CAREER: Harnessing the Continuum for Big Data: Partial Differential Equations, Calculus of Variations, and Machine Learning

机器学习在日常生活中有着广泛的应用，比如自动驾驶汽车、医学图像分析和语音识别。近年来，机器学习算法的能力取得了巨大进步，在许多这些任务上，机器学习算法可以复制甚至超越人类的表现。然而，对于大规模机器学习何时能够很好地发现数据中有趣的模式和结构，以及何时无法做到这一点并过度拟合，我们仍然缺乏理论上的理解。这些问题最近在深度神经网络的对抗性黑客攻击中表现出来，这对数据隐私和安全至关重要的敏感应用程序构成了风险。该项目的目标是利用偏微分方程理论和变分学来研究机器学习和数据科学中的基础问题，并在强大的理论原理基础上开发新的、更高效的算法。该项目将通过以下方式推进数据科学教育：为高中学生开发基于图形的学习的年度暑期学校，指导明尼苏达大学天才青年数学计划的高中生，开发基于图形的学习的当前研究的研究生课程，并支持妇女和代表性不足的群体参与项目的各个方面。该项目的总体目标是利用偏微分方程（PDE）连续极限来解决离散机器学习问题，分析现有算法，开发具有更好性能保证的新算法，并在机器学习和偏微分方程之间建立新的联系。该项目的一个主要焦点是基于图的学习，其中图上的离散学习算法可以被解释为连续偏微分方程的离散化，这些偏微分方程的性质（例如，规律性和适定性）传达了关于学习问题的信息，并可以导致建立在强大理论原理基础上的新算法。特别是，该项目将(1)开发和研究一种新的算法，称为泊松学习，用于以非常低的标记率进行基于图的半监督学习；(2)在非线性偏微分方程的随机梯度下降（SGD）和粘度解之间建立新的联系，表明SGD具有在不适定问题中选择良好最小化的能力；(3)利用经典椭圆正则性理论的思想证明随机几何图上基于图的学习问题解的内Lipschitz正则性；(4)证明低标记率下基于图的学习的泛化界限，包括主动学习设置；(5)发展有向图学习问题的一致性理论，如排序算法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning has broad applications in everyday life, such as self-driving cars, medical image analysis, and speech recognition. Recent years have seen tremendous advances in the ability of machine learning algorithms to replicate and even exceed human performance on many of these tasks. However, we still lack a theoretical understanding of when large scale machine learning will work well to discover interesting patterns and structure in data, and when it will fail to do so and overfit. These problems have recently manifested in adversarial hacking of deep neural networks, which poses risks in sensitive applications where data privacy and security are paramount. The objective of this project is to use the theory of partial differential equations and the calculus of variations to study foundational problems in machine learning and data science, and develop new, more efficient, algorithms founded on strong theoretical principles. The project will advance data science education by developing an annual summer school for high school students on graph-based learning, mentoring high-school students from the University of Minnesota Talented Youth Mathematics Program, developing graduate courses on current research in graph-based learning, and supporting the participation of women and underrepresented groups in all aspects of the project.The overall goal of the project is to use partial differential equation (PDE) continuum limits for discrete machine learning problems to analyze existing algorithms, develop new algorithms with better performance guarantees, and make new connections between machine learning and PDEs. A major focus of the project is graph-based learning, where discrete learning algorithms on graphs can be interpreted as discretizations of continuum PDEs, and properties of those PDEs (e.g., regularity and well-posedness) convey information about the learning problem and can lead to new algorithms founded on strong theoretical principles. In particular, the project will (1) develop and study a new algorithm, called Poisson learning, for graph-based semi-supervised learning at very low labeling rates, (2) draw new connections between stochastic gradient descent (SGD) and viscosity solutions of nonlinear PDEs, showing that SGD has the ability to select good minimizers in ill-posed problems, (3) use ideas from classical elliptic regularity theory to prove interior Lipschitz regularity for solutions of graph-based learning problems on random geometric graphs, (4) prove generalization bounds for graph-based learning in low label-rate regimes, including the active learning setting, and (5) develop a consistency theory for learning problems on directed graphs, such as ranking algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.