CAREER: Deep Learning Based Scientific Computing: Mathematical Theory and Algorithms

深度学习在计算机视觉和自然语言处理任务上展示了卓越的高保真性能，这些任务彻底改变了制造业和社会生活。最近深度学习在科学问题上的应用也通过计算化学、材料科学、医学、免疫学、气候科学等推动了科学发现。了解深度学习算法的数学原理对于验证和改进这些算法至关重要，并将使科学家和工程师能够获得更可靠的预测并进行更好的风险评估。研究目标是开发一个系统的深度学习分析，作为基于深度学习的众多科学问题的理论基础；在各种应用领域中出现的高维和高度非线性偏微分方程的有效解的前沿算法也将提出并提供理论保证。所提出的基于深度学习的高维和高度非线性问题算法将有望极大地推动科学和工程许多领域中出现的复杂物理系统的最新模拟。深度学习的理论挑战主要是由于深度神经网络（dnn）的高度非线性性质。作为一种由非线性函数组成的函数参数化工具，深度神经网络是高度非线性的，需要高等数学才能完全理解。因此，为了更好地理解深度神经网络，迫切需要在数学方面取得新的进展。本课题的理论部分主要关注深度神经网络的近似和泛化能力。要回答的核心问题是DNN逼近是否克服或减轻了维数的诅咒，各种函数类的最佳逼近率是什么，以及如何表征以最优泛化误差界为目标的最先进的经验正则化方法训练的各种DNN的Rademacher复杂度。这个项目的计算部分集中于求解高维和高振荡的偏微分方程。该项目的具体方法是提出混合算法，结合深度学习算法和传统数值技术的优势，实现更高效的计算和更高的精度。关键思想是将深度学习求解器视为传统数值算法的先决条件。该项目中设计的算法也将在数值偏微分方程的深度学习包中实现，并公开提供。该项目的研究成果将通过会议、出版物（期刊论文和教科书）和新的数学深度学习课程传播给广大受众，特别是下一代计算科学家。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning has demonstrated remarkable, high fidelity performance on computer vision and natural language processing tasks that revolutionize manufacturing and social life. Recent applications of deep learning in scientific problems have also advanced scientific discovery via computational chemistry, materials science, medicine, immunology, climate sciences, etc. Understanding the mathematical principles of deep learning algorithms is crucial to validating and improving these algorithms, and will allow scientists and engineers to obtain more reliable predictions and perform a better risk assessment. The research goal is to develop a systematic deep learning analysis serving as the theoretical foundation of numerous scientific problems based on deep learning; cutting-edge algorithms for the efficient solutions of high-dimensional and highly nonlinear partial differential equations arising in various application domains will also be proposed with a theoretical guarantee. The proposed deep learning-based algorithms for high-dimensional and highly nonlinear problems will be expected to greatly advance the state-of-the-art simulations of complex physical systems arising in many fields in science and engineering. The theoretical challenges of deep learning are largely due to the highly non-linear nature of deep neural networks (DNNs). As a function parametrization tool formulated as compositions of non-linear functions, DNNs are highly non-linear and require advanced mathematics to fully understand. Therefore, there is a critical need for new advances in mathematics for a better understanding of DNNs. The theoretical part of this project mainly focuses on the approximation and generalization capacity of DNNs. The central questions to be answered are whether DNN approximation conquers or lessens the curse of dimensionality, what is the optimal approximation rate of various function classes, and how to characterize the Rademacher complexity of various DNNs trained with state-of-the-art empirical regularization methods aiming at optimal generalization error bound. The computational part of this project concentrates on solving high dimensional and highly oscillatory partial differential equations. The specific approach of this project is to propose hybrid algorithms that combine the advantage of deep learning algorithms and traditional numerical techniques for more efficient computation and higher accuracy. The key idea is to treat deep learning solvers as a preconditioner of traditional numerical algorithms. The algorithms designed in the project will also be implemented in deep learning packages for numerical PDEs and made publicly available. Research outcomes of this project will be disseminated through conferences, publications (journal papers and textbooks), and new mathematical deep learning courses to a broad audience, especially for the next generation of computational scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.