CAREER: Deep Learning Based Scientific Computing: Mathematical Theory and Algorithms

深度学习在计算机视觉和自然语言处理任务中表现出显著的高保真性能，这些任务给制造业和社会生活带来了革命性的变化。近年来，深度学习在科学问题中的应用也推动了计算化学、材料科学、医学、免疫学、气候科学等科学发现的发展。理解深度学习算法的数学原理对于验证和改进这些算法至关重要，将使科学家和工程师获得更可靠的预测和更好的风险评估。研究的目标是开发一个系统的深度学习分析，作为基于深度学习的众多科学问题的理论基础；还将提出解决各种应用领域中出现的高维和高度非线性偏微分方程组的有效解的前沿算法，并提供理论保证。所提出的基于深度学习的高维和高度非线性问题的算法将极大地促进科学和工程中许多领域出现的复杂物理系统的最新模拟。深度学习的理论挑战很大程度上源于深度神经网络(DNN)的高度非线性。作为一种由非线性函数组合而成的函数参数化工具，DNN是高度非线性的，需要高等数学才能完全理解。因此，迫切需要在数学方面取得新的进展，以便更好地理解DNN。本项目的理论部分主要关注DNN的逼近和泛化能力。需要回答的核心问题是DNN逼近是否克服或减少了维度灾难，各种函数类的最优逼近速度是多少，以及如何表征以最优泛化误差界为目标的各种经验正则化方法训练的DNN的Rademacher复杂性。这个项目的计算部分集中在求解高维和高度振荡的偏微分方程组。该项目的具体方法是提出结合深度学习算法和传统数值技术的优点的混合算法，以实现更高效的计算和更高的精度。其关键思想是将深度学习求解器作为传统数值算法的预条件。该项目中设计的算法还将在数值PDE的深度学习包中实现，并公开可用。该项目的研究成果将通过会议、出版物(期刊论文和教科书)和新的数学深度学习课程向广大受众传播，特别是针对下一代计算科学家。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Deep learning has demonstrated remarkable, high fidelity performance on computer vision and natural language processing tasks that revolutionize manufacturing and social life. Recent applications of deep learning in scientific problems have also advanced scientific discovery via computational chemistry, materials science, medicine, immunology, climate sciences, etc. Understanding the mathematical principles of deep learning algorithms is crucial to validating and improving these algorithms, and will allow scientists and engineers to obtain more reliable predictions and perform a better risk assessment. The research goal is to develop a systematic deep learning analysis serving as the theoretical foundation of numerous scientific problems based on deep learning; cutting-edge algorithms for the efficient solutions of high-dimensional and highly nonlinear partial differential equations arising in various application domains will also be proposed with a theoretical guarantee. The proposed deep learning-based algorithms for high-dimensional and highly nonlinear problems will be expected to greatly advance the state-of-the-art simulations of complex physical systems arising in many fields in science and engineering. The theoretical challenges of deep learning are largely due to the highly non-linear nature of deep neural networks (DNNs). As a function parametrization tool formulated as compositions of non-linear functions, DNNs are highly non-linear and require advanced mathematics to fully understand. Therefore, there is a critical need for new advances in mathematics for a better understanding of DNNs. The theoretical part of this project mainly focuses on the approximation and generalization capacity of DNNs. The central questions to be answered are whether DNN approximation conquers or lessens the curse of dimensionality, what is the optimal approximation rate of various function classes, and how to characterize the Rademacher complexity of various DNNs trained with state-of-the-art empirical regularization methods aiming at optimal generalization error bound. The computational part of this project concentrates on solving high dimensional and highly oscillatory partial differential equations. The specific approach of this project is to propose hybrid algorithms that combine the advantage of deep learning algorithms and traditional numerical techniques for more efficient computation and higher accuracy. The key idea is to treat deep learning solvers as a preconditioner of traditional numerical algorithms. The algorithms designed in the project will also be implemented in deep learning packages for numerical PDEs and made publicly available. Research outcomes of this project will be disseminated through conferences, publications (journal papers and textbooks), and new mathematical deep learning courses to a broad audience, especially for the next generation of computational scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.