GPU Acceleration of Multigrid Preconditioned Conjugate Gradient Solver on Block-Structured Cartesian Grid
GPU Acceleration of Multigrid Preconditioned Conjugate Gradient Solver on Block-Structured Cartesian Grid
复制标题
DOI：
10.1145/3432261.3432273
复制
发表时间：
2021-01
期刊：
影响因子：
--
通讯作者：
Naoyuki Onodera;Y. Idomura;Yuta Hasegawa;S. Yamashita;T. Shimokawabe;T. Aoki
中科院分区：
文献类型：
--
作者：
Naoyuki Onodera;Y. Idomura;Yuta Hasegawa;S. Yamashita;T. Shimokawabe;T. Aoki
针对两相流计算流体力学程序JUPITER中的压力泊松方程，开发了一种多重网格预处理共轭梯度（MG-CG）求解器。JUPITER代码被重新设计，以实现高效的CFD模拟，包括复杂的边界和对象的基础上，块结构的笛卡尔网格系统。代码是用CUDA编写的，并经过调整，以在基于GPU的超级计算机上实现高性能。MG-CG求解器的主要内核实现了超过90%的车顶线性能。MG预处理器是基于几何MG方法构造的，具有一个三阶段的V-循环，并在每个阶段应用一个红-黑SOR（RB-SOR）平滑器及其变体与缓存重用优化（CR-SOR）。对核反应堆燃料棒束内的两相流动进行了数值实验。由于采用了块结构的数据格式，在不降低性能的情况下，去除了燃料细棒内部的网格，网格总数减少到2.26 × 109，约为原笛卡尔网格的70%。带有RB-SOR和CR-SOR平滑器的MG-CG求解器将迭代次数减少到原始预处理CG方法的15%和9%以下，分别获得3.1倍和5.9倍的加速比。在强缩放测试中，具有CR-SOR平滑器的MG-CG求解器在64和256 GPU之间加速2.1倍。所获得的性能表明，为块结构网格设计的MG-CG求解器是高效的，并且能够在基于GPU的超级计算机上进行大规模的两相流模拟。
We develop a multigrid preconditioned conjugate gradient (MG-CG) solver for the pressure Poisson equation in a two-phase flow CFD code JUPITER. The JUPITER code is redesigned to realize efficient CFD simulations including complex boundaries and objects based on a block-structured Cartesian grid system. The code is written in CUDA, and is tuned to achieve high performance on GPU based supercomputers. The main kernels of the MG-CG solver achieve more than 90% of the roofline performance. The MG preconditioner is constructed based on the geometric MG method with a three-stage V-cycle, and a red-black SOR (RB-SOR) smoother and its variant with cache-reuse optimization (CR-SOR) are applied at each stage. The numerical experiments are conducted for two-phase flows in a fuel bundle of a nuclear reactor. Thanks to the block-structured data format, grids inside fuel pins are removed without performance degradation, and the total number of grids is reduced to 2.26 × 109, which is about 70% of the original Cartesian grid. The MG-CG solvers with the RB-SOR and CR-SOR smoothers reduce the number of iterations to less than 15% and 9% of the original preconditioned CG method, leading to 3.1- and 5.9-times speedups, respectively. In the strong scaling test, the MG-CG solver with the CR-SOR smoother is accelerated by 2.1 times between 64 and 256 GPUs. The obtained performance indicates that the MG-CG solver designed for the block-structured grid is highly efficient and enables large-scale simulations of two-phase flows on GPU based supercomputers.