Solving High Frequency and Multi-Scale PDEs with Gaussian Processes
Solving High Frequency and Multi-Scale PDEs with Gaussian Processes
复制标题
DOI：
10.48550/arxiv.2311.04465
复制
发表时间：
2023-11
期刊：
影响因子：
--
通讯作者：
Shikai Fang;Madison Cooley;Da Long;Shibo Li;R. Kirby;Shandian Zhe
中科院分区：
文献类型：
--
作者：
Shikai Fang;Madison Cooley;Da Long;Shibo Li;R. Kirby;Shandian Zhe
基于机器学习的求解器在物理模拟和科学计算中引起了广泛关注，一个突出的例子是物理信息神经网络（PINN）。然而，PINN通常难以解决高频和多尺度PDE，这可能是由于神经网络训练期间的频谱偏差。为了解决这个问题，我们求助于高斯过程（GP）框架。为了灵活地捕获主频率，我们使用学生$t$混合或高斯混合来建模PDE解决方案的功率谱。我们应用逆傅立叶变换来获得协方差函数（通过Wiener-Khinchin定理）。从高斯混合谱导出的协方差对应于已知的谱混合核。接下来，我们在对数域中估计混合权重，这相当于放置Jeffreys先验。它会自动诱导稀疏性，修剪过多的频率，并将剩余的频率调整为地面真实值。第三，为了在大量配置点上实现高效和可扩展的计算，这对于捕获高频率至关重要，我们将配置点放置在网格上，并在每个输入维度上乘以我们的协方差函数。我们使用GP条件平均来预测解及其导数，以便拟合边界条件和方程本身。因此，我们可以在协方差矩阵中导出Kronecker乘积结构。我们使用克罗内克产品的性质和多线性代数，以提高计算效率和可扩展性，没有低秩近似。在系统的实验中，我们展示了我们的方法的优势。代码发布于\url{https：//github.com/xuangu-fang/Gaussian-Process-Slover-for-High-Freq-PDE}。
Machine learning based solvers have garnered much attention in physical simulation and scientific computing, with a prominent example, physics-informed neural networks (PINNs). However, PINNs often struggle to solve high-frequency and multi-scale PDEs, which can be due to spectral bias during neural network training. To address this problem, we resort to the Gaussian process (GP) framework. To flexibly capture the dominant frequencies, we model the power spectrum of the PDE solution with a student $t$ mixture or Gaussian mixture. We apply the inverse Fourier transform to obtain the covariance function (by Wiener-Khinchin theorem). The covariance derived from the Gaussian mixture spectrum corresponds to the known spectral mixture kernel. Next, we estimate the mixture weights in the log domain, which we show is equivalent to placing a Jeffreys prior. It automatically induces sparsity, prunes excessive frequencies, and adjusts the remaining toward the ground truth. Third, to enable efficient and scalable computation on massive collocation points, which are critical to capture high frequencies, we place the collocation points on a grid, and multiply our covariance function at each input dimension. We use the GP conditional mean to predict the solution and its derivatives so as to fit the boundary condition and the equation itself. As a result, we can derive a Kronecker product structure in the covariance matrix. We use Kronecker product properties and multilinear algebra to promote computational efficiency and scalability, without low-rank approximations. We show the advantage of our method in systematic experiments. The code is released at \url{https://github.com/xuangu-fang/Gaussian-Process-Slover-for-High-Freq-PDE}.