Stability, Accuracy and Efficiency in Hybrid Finite Element / Neural Network Simulations

偏微分方程组的求解是数值分析的中心课题，也是科学和工程中不可缺少的工具。现有的方法，如有限元，可以在许多应用中提供高效和稳健的解。深度神经网络是最近几年出现的一种替代方法，取得了令人振奋的结果。然而，完全或部分基于神经网络的技术目前缺乏现有方法可用的数学保证和见解。它们在应用中的相对性能和实用健壮性目前也不清楚。我们将致力于将有限元和深度神经网络相结合的数值技术的数学理论，用于求解偏微分方程组。我们的假设是，这种混合方法可以提供比单独使用其中任何一种方法在计算上更高效、更准确的解决方案。我们考虑了用神经网络表示细尺度行为的Navier-Stokes方程，该方程没有被有限元求解。这些网络使用高分辨率参考数据进行训练。因此，我们不会像在PINN中那样追求对解决方案的物理或数学限制，而是将其视为一个重要但正交的研究方向。拟议项目的目标是开发严格的数学分析，但我们认为通过实施来研究我们结果的实用性也是同样重要的。因此，混合流体流动解算器的研究代码将被实施并公开提供。我们建立在最近的工作的基础上，这些工作表明，使用为分析有限元方法而开发的工具可以对深度神经网络进行数学分析。我们将把这些结果推广到求解Navier-Stokes方程的混合数值时间步长格式，并考虑实际相关的设置。此外，我们将现有的结果扩展到最先进的神经网络结构，例如变压器。它们是实践中使用的最强大的体系结构之一，同时非常适合科学计算和数学分析。我们将解决的核心问题是混合模拟的稳定性和准确性，即它们保持有界，并且神经网络能够提高精度。对于混合求解器，这要求神经网络对于允许的输入是稳定的，但也需要耦合到保持稳定的有限元部分。第二，我们将探索自适应解决方案，其中使用基于后验或神经网络的误差估计来改进解决方案，如有必要，以满足预定义的误差标准。我们相信，在拟议的项目中获得的结果也将与基于神经网络的模拟的更完整的理论相关。

The solution of partial differential equations is a central subject of numerical analysis and an indispensable tool in science and engineering. Existing approaches, such as finite elements, can provide solutions efficiently and robustly in many applications. Deep neural networks emerged in the last few years as an alternative approach with promising results. Techniques that are completely or partially based on neural networks, however, currently lack the mathematical guarantees and insights available for established approaches. Their relative performance and practical robustness in applications is also unclear at the moment. We will work towards a mathematical theory of numerical techniques that combine finite elements and deep neural networks for the solution of partial differential equations. Our hypothesis is that such a hybrid approach can provide a computationally more efficient and more accurate solution than either approach alone. We consider the Navier-Stokes equations with the neural networks representing fine scale behavior not resolved by finite elements. The networks are trained using high-resolution reference data. We will therefore not pursue physical or mathematical constraints on the solutions, as in PINNs, and consider it an important but orthogonal research direction. The objective of the proposed project is to develop mathematically rigorous analyses, but we consider it also as important to study the practicality of our results through implementations. A research code for hybrid fluid flow solvers will therefore be implemented and made publicly available. We build on recent work that showed that the mathematical analysis of deep neural networks is possible using tools developed for the analysis of finite element methods. We will extend these results to hybrid numerical time stepping schemes for the Navier-Stokes equations and consider practically relevant setups. Further we extend existing results to state-of-the-art neural network architectures, e.g. transformers. These are one of the most powerful architectures used in practice and at the same time well suited for scientific computing and a mathematical analysis. Central questions we will address are stability and accuracy of the hybrid simulations, i.e. that they remain bounded and that a neural network is able to improve the accuracy. For a hybrid solver, this requires, among other things, neural networks that are stable for admissible inputs but also a coupling to the finite element part that preserves stability. Second, we will explore adaptive solution schemes where a posteriori or neural network-based error estimates are used to refine a solution if necessary, to meet predefined error criteria. We believe that the results obtained in the proposed project will also be of relevance for a more complete theory for neural network-based simulations.