Multigrid Methods and Machine Learning

该项目的目标是将多重网格（MG）方法和机器学习（ML）的先进工具融合在一起，开发一类新型数值技术，针对物理、生物和社会科学中新兴的数据密集型应用。 多重网格方法，包括几何和代数多重网格（GMG 和 AMG）方法，是求解科学和工程计算中产生的线性和非线性代数方程组的有效工具。 另一方面，机器学习（ML）技术取得了显着进步，尤其是卷积神经网络（CNN），在图像分类和处理等许多领域都有成功的应用。 拟议的项目是探索这两种不同技术之间的相似性和差异，以便开发更有效的多重网格方法以及更有效的深度学习模型。 现有丰富的多重网格方法理论有望为深度神经网络的理论理解提供新的思路，而大量且不断增长的深度学习文献中使用的大量经验技术可用于设计具有更广泛应用的通用多重网格方法。 这个跨学科的研究项目预计将对科学计算界和人工智能行业产生直接影响。更具体地说，MG 和 CNN 在多级层次结构的使用以及许多技术组件的使用方面相似，例如平滑器 (MG) 与卷积 (CNN)、限制 (MG) 与跨步卷积 (CNN)。但它们也有一些主要区别：CNN 有多个卷积通道需要训练，而 MG 通常有一个先验的平滑器。这种关系激发了新的多重网格方法的设计，这些方法具有更通用的平滑器和限制，这些方法以不同的方式进行主题训练，因此，多重网格方法在应用于不同的实际问题时将变得更具适应性和鲁棒性。 易于理解的 MG 结构和理论可以用来理解和改进现有的深度学习模型，例如残差神经网络。 此外，还将研究 MG 中使用的多级迭代技术，以加速随机梯度下降法，该方法现在是机器学习中大多数深度神经网络的标准训练算法。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project is to merge advanced tools from multigrid (MG) methods and machine learning (ML) towards the development of a novel class of numerical techniques targeting the data intensive applications emerging in physical, biological and social sciences. Multigrid methods, including both geometric and algebraic multigrid (GMG and AMG) methods, are effective tools for solving linear as well as nonlinear algebraic system of equations arising from scientific and engineering computing. On the other hand, there is a significant advancement in machine learning (ML) techniques, especially convolutional neural networks (CNN), which have successful applications in many areas such as image classification and processing. The proposed project is to explore the resemblances and differences between these two different technologies so that more efficient multigrid methods as well more efficient deep learning models are developed. The existing rich theory of multigrid method is expected to shed new light to the theoretical understanding of deep neural networks whereas the numerous empirical techniques used in the vast and ever-growning deep learning literature can be used to design general multigrid methods with wider range of applications. This interdisciplinary research project is expected to have a direct impact to both the scientific computing community and the artificial intelligence industry.More specifically, MG and CNN are similar for the use of multilevel hierarchy and the use of many technical components such as smoothers (MG) versus convolutions (CNN), restriction (MG) versus convolution with stride (CNN). But they also have some major differences: CNN has multiple channels of convolutions to be trained whereas MG often has one single smoother given a priori. Such relationships motivate the design of new multigrid methods with more general smoothers and restrictions that are subject training in different ways and, as a result, multigrid methods will become more adaptive and robust in its application to different practical problems. The well-understood MG structure and theory can be adapted to understand and improve the existing deep learning model such as residual neural networks. Furthermore, multilevel iterative techniques used in MG will also be investigated to speed up the stochastic gradient descent method that is now the standard training algorithm for most deep neural networks in machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.