Leveraging MPI RMA to optimize halo-swapping communications in MONC on Cray machines
Leveraging MPI RMA to optimize halo-swapping communications in MONC on Cray machines
复制标题
DOI：
10.1002/cpe.5008
复制
发表时间：
2019-08-25
影响因子：
2
通讯作者：
Weiland, Michele
中科院分区：
文献类型：
--
作者：
Brown, Nick;Bareford, Michael;Weiland, Michele
远程内存访问（RMA），也称为单边通信，提供了一种直接对其他进程的内存进行读写的方法，而无需发出显式的消息传递式通信调用。先前的研究得出结论，MPI RMA相较于传统的MPI点对点（P2P）可以提供更高的通信性能，但这些结论是基于合成基准测试而非实际代码。在这项工作中，我们用MPI RMA替换了英国气象局NERC云模型（一个用于模拟大气的成熟代码）中现有的非阻塞P2P通信调用。我们详细描述了我们的方法，并讨论了为确保正确性和性能所采取的选项。实验在ARCHER（一台Cray XC30）和Cirrus（一台SGI ICE机器）上进行。我们在ARCHER上证明，通过使用RMA，在多达32768个核心上，每个时间步的通信时间可减少5%到10%，在整个运行过程（有许多时间步）中，与Cray上的P2P相比，性能有显著提高。然而，RMA并非万能之策，在将RMA调用集成到现有代码中时存在挑战：要实现良好的性能必须进行重要的优化，而且库支持并非普遍成熟，Cirrus的情况就是如此。在本文中，我们在一个实际代码的背景下，讨论了将P2P转换为RMA的经验教训，探索了性能和扩展方面的挑战，并详细对比了不同的RMA同步方法。
Remote Memory Access (RMA), also known as single-sided communications, provides a way for reading and writing directly into the memory of other processes without having to issue explicit message passing style communication calls. Previous studies have concluded that MPI RMA can provide increased communication performance over traditional MPI Point to Point (P2P), but these are based on synthetic benchmarks rather than real-world codes. In this work, we replace the existing non-blocking P2P communication calls in the Met Office NERC Cloud model, a mature code for modeling the atmosphere, with MPI RMA. We describe our approach in detail and discuss the options taken for correctness and performance. Experiments are performed on ARCHER, a Cray XC30, and Cirrus, an SGI ICE machine. We demonstrate on ARCHER that, by using RMA, we can obtain between a 5% and 10% reduction in communication time at each timestep on up to 32768 cores, which over the entirety of a run (with many timesteps) results in a significant improvement in performance compared to P2P on the Cray. However, RMA is not a silver bullet, and there are challenges when integrating RMA calls into existing codes: important optimizations are necessary to achieve good performance and library support is not universally mature, as is the case on Cirrus. In this paper, we discuss, in the context of a real-world code, the lessons learned converting P2P to RMA, explore performance and scaling challenges, and contrast alternative RMA synchronization approaches in detail.