Stability, Accuracy and Efficiency in Hybrid Finite Element / Neural Network Simulations

偏微分方程的解是数值分析的中心课题，也是科学和工程中不可缺少的工具。现有的方法，如有限元，可以在许多应用中提供有效和鲁棒的解决方案。深度神经网络在过去几年中作为一种替代方法出现，并取得了可喜的成果。然而，完全或部分基于神经网络的技术目前缺乏可用于既定方法的数学保证和见解。它们在应用中的相对性能和实际鲁棒性目前也不清楚。我们将致力于一个数学理论的数值技术，结合联合收割机有限元和深度神经网络的偏微分方程的解决方案。我们的假设是，这样的混合方法可以提供一个计算更有效，更准确的解决方案比单独的方法。我们考虑Navier-Stokes方程的神经网络表示精细尺度行为，而不是解决有限元。网络使用高分辨率参考数据进行训练。因此，我们将不追求物理或数学的限制的解决方案，在PINN，并认为这是一个重要的，但正交的研究方向。拟议项目的目标是开发数学上严格的分析，但我们认为，通过实施研究我们的结果的实用性也很重要。因此，混合流体流动求解器的研究代码将被实施并公开提供。 我们建立在最近的工作基础上，这些工作表明，使用为有限元方法分析开发的工具，可以对深度神经网络进行数学分析。我们将把这些结果推广到Navier-Stokes方程的混合数值时间步进格式，并考虑实际相关的设置。此外，我们将现有的结果扩展到最先进的神经网络架构，例如变压器。这些是在实践中使用的最强大的架构之一，同时非常适合科学计算和数学分析。我们将解决的中心问题是混合模拟的稳定性和准确性，即它们保持有界，并且神经网络能够提高准确性。对于混合求解器，这需要，除其他事项外，神经网络是稳定的容许输入，但也耦合到有限元部分，保持稳定性。其次，我们将探讨自适应解决方案，其中后验或基于神经网络的误差估计用于在必要时改进解决方案，以满足预定义的误差标准。我们相信，在拟议的项目中获得的结果也将是一个更完整的基于神经网络的模拟理论的相关性。

The solution of partial differential equations is a central subject of numerical analysis and an indispensable tool in science and engineering. Existing approaches, such as finite elements, can provide solutions efficiently and robustly in many applications. Deep neural networks emerged in the last few years as an alternative approach with promising results. Techniques that are completely or partially based on neural networks, however, currently lack the mathematical guarantees and insights available for established approaches. Their relative performance and practical robustness in applications is also unclear at the moment. We will work towards a mathematical theory of numerical techniques that combine finite elements and deep neural networks for the solution of partial differential equations. Our hypothesis is that such a hybrid approach can provide a computationally more efficient and more accurate solution than either approach alone. We consider the Navier-Stokes equations with the neural networks representing fine scale behavior not resolved by finite elements. The networks are trained using high-resolution reference data. We will therefore not pursue physical or mathematical constraints on the solutions, as in PINNs, and consider it an important but orthogonal research direction. The objective of the proposed project is to develop mathematically rigorous analyses, but we consider it also as important to study the practicality of our results through implementations. A research code for hybrid fluid flow solvers will therefore be implemented and made publicly available. We build on recent work that showed that the mathematical analysis of deep neural networks is possible using tools developed for the analysis of finite element methods. We will extend these results to hybrid numerical time stepping schemes for the Navier-Stokes equations and consider practically relevant setups. Further we extend existing results to state-of-the-art neural network architectures, e.g. transformers. These are one of the most powerful architectures used in practice and at the same time well suited for scientific computing and a mathematical analysis. Central questions we will address are stability and accuracy of the hybrid simulations, i.e. that they remain bounded and that a neural network is able to improve the accuracy. For a hybrid solver, this requires, among other things, neural networks that are stable for admissible inputs but also a coupling to the finite element part that preserves stability. Second, we will explore adaptive solution schemes where a posteriori or neural network-based error estimates are used to refine a solution if necessary, to meet predefined error criteria. We believe that the results obtained in the proposed project will also be of relevance for a more complete theory for neural network-based simulations.