Multigrid Methods and Machine Learning

多重网格方法和机器学习

• 批准号: 1819157
• 负责人: Jinchao Xu
• 金额: $ 35万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1819157&HistoricalAwards=false
• 关键词: Multigrid; Methods; Machine; Learning

The goal of this project is to merge advanced tools from multigrid (MG) methods and machine learning (ML) towards the development of a novel class of numerical techniques targeting the data intensive applications emerging in physical, biological and social sciences. Multigrid methods, including both geometric and algebraic multigrid (GMG and AMG) methods, are effective tools for solving linear as well as nonlinear algebraic system of equations arising from scientific and engineering computing. On the other hand, there is a significant advancement in machine learning (ML) techniques, especially convolutional neural networks (CNN), which have successful applications in many areas such as image classification and processing. The proposed project is to explore the resemblances and differences between these two different technologies so that more efficient multigrid methods as well more efficient deep learning models are developed. The existing rich theory of multigrid method is expected to shed new light to the theoretical understanding of deep neural networks whereas the numerous empirical techniques used in the vast and ever-growning deep learning literature can be used to design general multigrid methods with wider range of applications. This interdisciplinary research project is expected to have a direct impact to both the scientific computing community and the artificial intelligence industry.More specifically, MG and CNN are similar for the use of multilevel hierarchy and the use of many technical components such as smoothers (MG) versus convolutions (CNN), restriction (MG) versus convolution with stride (CNN). But they also have some major differences: CNN has multiple channels of convolutions to be trained whereas MG often has one single smoother given a priori. Such relationships motivate the design of new multigrid methods with more general smoothers and restrictions that are subject training in different ways and, as a result, multigrid methods will become more adaptive and robust in its application to different practical problems. The well-understood MG structure and theory can be adapted to understand and improve the existing deep learning model such as residual neural networks. Furthermore, multilevel iterative techniques used in MG will also be investigated to speed up the stochastic gradient descent method that is now the standard training algorithm for most deep neural networks in machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的目标是将多重网格（MG）方法和机器学习（ML）的高级工具合并到一类新的数值技术的开发中，这些技术针对物理，生物和社会科学中出现的数据密集型应用。 多重网格方法，包括几何多重网格方法和代数多重网格方法（GMG和AMG），是求解科学和工程计算中的线性和非线性代数方程组的有效工具。 另一方面，机器学习（ML）技术取得了重大进展，特别是卷积神经网络（CNN），它在图像分类和处理等许多领域都有成功的应用。 该项目旨在探索这两种不同技术之间的相似性和差异，以便开发更有效的多重网格方法以及更有效的深度学习模型。 现有的丰富的多重网格方法理论有望为深度神经网络的理论理解提供新的思路，而大量不断增长的深度学习文献中使用的众多经验技术可用于设计具有更广泛应用的通用多重网格方法。 这个跨学科的研究项目预计将对科学计算社区和人工智能行业产生直接影响。更具体地说，MG和CNN在使用多级层次结构和使用许多技术组件方面是相似的，例如平滑器（MG）与卷积（CNN），限制（MG）与步幅卷积（CNN）。但它们也有一些主要的区别：CNN有多个卷积通道要训练，而MG通常只有一个先验平滑器。这种关系促使设计新的多重网格方法，更一般的平滑和限制，以不同的方式进行主题训练，因此，多重网格方法将变得更加适应和强大的应用到不同的实际问题。 已经被充分理解的MG结构和理论可以被用来理解和改进现有的深度学习模型，如残差神经网络。 此外，还将研究MG中使用的多级迭代技术，以加快随机梯度下降方法，该方法现在是机器学习中大多数深度神经网络的标准训练算法。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

A posteriori error estimates of finite element methods by preconditioning

• DOI: 10.1016/j.camwa.2020.08.001
• 发表时间: 2020-02
• 期刊: Comput. Math. Appl.
• 作者: Yuwen Li;L. Zikatanov

Residual-based a posteriori error estimates of mixed methods for a three-field Biot’s consolidation model

• DOI: 10.1093/imanum/draa074
• 发表时间: 2019-11
• 期刊: IMA Journal of Numerical Analysis
• 影响因子: 2.1
• 作者: Yuwen Li;L. Zikatanov

Robust Preconditioners for a New Stabilized Discretization of the Poroelastic Equations

用于多孔弹性方程新稳定离散化的鲁棒预条件子

• DOI: 10.1137/19m1261250
• 发表时间: 2020
• 期刊: SIAM Journal on Scientific Computing
• 影响因子: 3.1
• 作者: Adler, J. H.;Gaspar, F. J.;Hu, X.;Ohm, P.;Rodrigo, C.;Zikatanov, L. T.
• 通讯作者: Zikatanov, L. T.

Discrete trace theorems and energy minimizing spring embeddings of planar graphs

平面图的离散迹定理和能量最小化弹簧嵌入

• DOI: 10.1016/j.laa.2020.08.035
• 发表时间: 2021
• 期刊: Linear Algebra and its Applications
• 影响因子: 1.1
• 作者: Urschel, John C.;Zikatanov, Ludmil T.
• 通讯作者: Zikatanov, Ludmil T.

Uniform Stability and Error Analysis for Some Discontinuous Galerkin Methods

• DOI: 10.4208/jcm.2003-m2018-0223
• 发表时间: 2018-05
• 期刊: Journal of Computational Mathematics
• 影响因子: 0.9
• 作者: Q. Hong;Jinchao Xu