CAREER: Deep Learning Based Scientific Computing: Mathematical Theory and Algorithms

深度学习在计算机视觉和自然语言处理任务上表现出了卓越的高保真性能，这些任务彻底改变了制造业和社会生活。最近深度学习在科学问题中的应用也通过计算化学，材料科学，医学，免疫学，气候科学等推进了科学发现，理解深度学习算法的数学原理对于验证和改进这些算法至关重要，并将使科学家和工程师能够获得更可靠的预测并进行更好的风险评估。研究目标是开发系统的深度学习分析，作为基于深度学习的众多科学问题的理论基础;还将提出在各种应用领域中产生的高维和高度非线性偏微分方程的有效解决方案的尖端算法，并提供理论保证。针对高维和高度非线性问题提出的基于深度学习的算法将有望大大推进科学和工程许多领域中出现的复杂物理系统的最先进模拟。深度学习的理论挑战主要是由于深度神经网络（DNN）的高度非线性性质。作为一种函数参数化工具，DNN是高度非线性的，需要高等数学才能完全理解。因此，迫切需要在数学方面取得新的进展，以便更好地理解DNN。该项目的理论部分主要集中在DNN的近似和泛化能力。需要回答的核心问题是DNN近似是否克服或减轻维数灾难，各种函数类的最佳近似率是多少，以及如何表征各种DNN的Rademacher复杂度，这些DNN是用最先进的经验正则化方法训练的，旨在获得最佳泛化误差界。这个项目的计算部分集中在解决高维和高振荡偏微分方程。该项目的具体做法是提出混合算法，将深度学习算法和传统数值技术的优势结合联合收割机，实现更高效的计算和更高的精度。其核心思想是将深度学习求解器视为传统数值算法的预处理器。该项目中设计的算法也将在数值偏微分方程的深度学习包中实现，并公开提供。该项目的研究成果将通过会议、出版物（期刊论文和教科书）和新的数学深度学习课程向广大受众传播，特别是面向下一代计算科学家。该奖项反映了NSF的法定使命，通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning has demonstrated remarkable, high fidelity performance on computer vision and natural language processing tasks that revolutionize manufacturing and social life. Recent applications of deep learning in scientific problems have also advanced scientific discovery via computational chemistry, materials science, medicine, immunology, climate sciences, etc. Understanding the mathematical principles of deep learning algorithms is crucial to validating and improving these algorithms, and will allow scientists and engineers to obtain more reliable predictions and perform a better risk assessment. The research goal is to develop a systematic deep learning analysis serving as the theoretical foundation of numerous scientific problems based on deep learning; cutting-edge algorithms for the efficient solutions of high-dimensional and highly nonlinear partial differential equations arising in various application domains will also be proposed with a theoretical guarantee. The proposed deep learning-based algorithms for high-dimensional and highly nonlinear problems will be expected to greatly advance the state-of-the-art simulations of complex physical systems arising in many fields in science and engineering. The theoretical challenges of deep learning are largely due to the highly non-linear nature of deep neural networks (DNNs). As a function parametrization tool formulated as compositions of non-linear functions, DNNs are highly non-linear and require advanced mathematics to fully understand. Therefore, there is a critical need for new advances in mathematics for a better understanding of DNNs. The theoretical part of this project mainly focuses on the approximation and generalization capacity of DNNs. The central questions to be answered are whether DNN approximation conquers or lessens the curse of dimensionality, what is the optimal approximation rate of various function classes, and how to characterize the Rademacher complexity of various DNNs trained with state-of-the-art empirical regularization methods aiming at optimal generalization error bound. The computational part of this project concentrates on solving high dimensional and highly oscillatory partial differential equations. The specific approach of this project is to propose hybrid algorithms that combine the advantage of deep learning algorithms and traditional numerical techniques for more efficient computation and higher accuracy. The key idea is to treat deep learning solvers as a preconditioner of traditional numerical algorithms. The algorithms designed in the project will also be implemented in deep learning packages for numerical PDEs and made publicly available. Research outcomes of this project will be disseminated through conferences, publications (journal papers and textbooks), and new mathematical deep learning courses to a broad audience, especially for the next generation of computational scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.