Toward Better Simulation of MPI Applications on Ethernet/TCP Networks
Toward Better Simulation of MPI Applications on Ethernet/TCP Networks
复制标题
更好地模拟以太网/TCP 网络上的 MPI 应用
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
B. Videau
中科院分区：
文献类型：
--
作者：
Paul Bédaride;A. Degomme;S. Genaud;Arnaud Legrand;George S. Markomanolis;M. Quinson;Mark Stillwell;F. Suter;B. Videau
绩效预测和分析的仿真和建模对于开发和维护HPC代码至关重要，该代码有望扩展下一代Exascale系统，并且正确建模网络行为对于创建现实模拟至关重要。在本文中，我们描述了基于流的混合网络模型的实现，该模型是诸如网络拓扑和争论之类的因素，这些因素通常被其他方法忽略。我们专注于大规模，以太网连接的系统，因为这些系统目前占前500个指数的37.8％，并且随着高速10和100GBE的更高速度，该份额有望增加。欧洲Mont-Blanc项目通过开发具有低功率嵌入式设备的原型系统来研究Exascale计算，它使用基于以太网的互连。我们的模型是在SMPI中实现的，SMPI是一种开源MPI实现，将实际应用程序连接到Simgrid Simulation框架。 SMPI根据OpenMPI和MPICH的当前版本提供了集体通信的实现。 SMPI和SIMGRID还提供了简化大规模系统模拟的方法，包括阴影执行，内存折叠以及对在线和离线的支持（即验尸）仿真。我们通过将SMPI与现实世界实验的轨迹以及使用其他已建立的网络模型获得的模型进行比较来验证我们的模型。我们的研究表明，SMPI的预测能力始终比基于经典的LOGP模型更好，包括已建立的HPC基准和实际应用，包括广泛的情况。
Simulation and modeling for performance prediction and profiling is essential for developing and maintaining HPC code that is expected to scale for next-generation exascale systems, and correctly modeling network behavior is essential for creating realistic simulations. In this article we describe an implementation of a flow-based hybrid network model that accounts for factors such as network topology and contention, which are commonly ignored by other approaches. We focus on large-scale, Ethernet-connected systems, as these currently compose 37.8 % of the TOP500 index, and this share is expected to increase as higher-speed 10 and 100GbE become more available. The European Mont-Blanc project, which studies exascale computing by developing prototype systems with low-power embedded devices, uses Ethernet-based interconnect. Our model is implemented within SMPI, an open-source MPI implementation that connects real applications to the SimGrid simulation framework. SMPI provides implementations of collective communications based on current versions of both OpenMPI and MPICH. SMPI and SimGrid also provide methods for easing the simulation of large-scale systems, including shadow execution, memory folding, and support for both online and offline (i.e., post-mortem) simulation. We validate our proposed model by comparing traces produced by SMPI with those from real world experiments, as well as with those obtained using other established network models. Our study shows that SMPI has a consistently better predictive power than classical LogP-based models for a wide range of scenarios including both established HPC benchmarks and real applications.