CDS&E: Robust Symmetry-Preserving Machine Learning: Theory and Application

深度神经网络（dnn）一直是数据科学和工程最新进展背后的主要推动力。深度神经网络研究的一个新兴主题是利用学习问题的内在结构，如对称性，来提高小数据体制下深度神经网络的数据效率。最近关于保持对称性的机器学习的研究通常是在对称变换完美的理想环境下进行的，然而，在现实中，它们通常受到各种信号变形源的“污染”。该项目的目的是严格测量和保证一般保持对称性的深度神经网络的变形鲁棒性，并量化其产生的性能增益。研究结果有望促进对鲁棒几何深度学习的理解，从计算机视觉到有限数据的科学计算的各种应用。该项目将为本科生和研究生提供应用数学、工程和数据科学方面的跨学科培训。该项目的首要主题是利用微分几何、应用谐波分析和应用概率中的数学工具来提高机器学习模型的统计效率。特别强调了对广义特征域上任意李群表示的鲁棒对称性保持的严格分析和推广。此外，该项目旨在将对称性保持的思想扩展到深度分布学习，并提出了一个统一的框架，用于数据高效生成具有内在结构的分布，包括但不限于群对称；提高的统计效率将通过样本复杂度分析严格量化。该项目开发的技术将广泛应用于不同学科，为下一代大数据计算和几何建模的数学工具提供基础构建模块。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep neural networks (DNNs) have been a major driving force behind recent advances in data science and engineering. An emerging theme in DNN research is to exploit the intrinsic structure of the learning problems, such as symmetry, to improve the data-efficiency of DNNs in the small-data regime. Recent work on symmetry-preserving machine learning typically studies it in the ideal setting where the symmetry transformations are perfect, whereas in reality, however, they are usually “contaminated” by various sources of signal deformation. The aim of this project is to rigorously measure and guarantee the deformation robustness of general symmetry-preserving DNNs, as well as quantifying their resulting performance gain. Results of the research are expected to advance understanding of robust geometric deep learning, with a diverse range of applications from computer vision to scientific computing with limited data. The project will provide interdisciplinary training in applied mathematics, engineering, and data science to undergraduate and graduate students. The overarching theme of the project is to leverage mathematical tools from differential geometry, applied harmonic analysis, and applied probability to improve the statistical-efficiency of machine learning models. Special emphasis has been placed on the rigorous analysis and promotion of robust symmetry-preservation that is broadly applicable to arbitrary Lie group representations on general feature fields. In addition, the project aims to extend the idea of symmetry-preservation to deep distribution learning, and proposes a unified framework for data-efficient generation of distributions with intrinsic structures including—but not limited to—group symmetry; the improved statistical efficiency will be rigorously quantified through sample complexity analysis. The techniques to be developed in this project will be widely applicable across different disciplines, providing fundamental building blocks for the next generation of mathematical tools for the computational and geometric modeling of Big Data.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.