Performance analysis on hybrid parallel programming on SMP cluster system

在本研究中，我们构建了一个PC集群，将多个基于smp的PC节点与各种网络接口连接起来。这样的SMP集群系统同时具有共享内存和分布式内存体系结构，并且有多种编程可能性，如消息传递、共享内存和它们的混合。我们的实验集群包含4路和2路SMP的奔腾- iii处理器和两种类型的互连网络，Myrinet800和快速以太网。我们评估和分析了混合MPI和OpenMP的混合编程性能，以及MPI仅针对NAS并行基准作为基本基准和SPAM（平滑粒子应用力学）粒子代码用于实际科学程序的消息传递编程。与初步估计相反，仅mpi编程在大多数程序中都取得了更好的性能。为了分析这些结果，我们根据编程风格，应用缓存命中率测量来研究缓存行为。因此，我们已经清除了混合程序，其中MPI用于节点间通信，OpenMP用于节点内多线程，这通常会破坏仅MPI程序上经过优化的缓存利用率。研究表明，数据访问模式对混合程序的性能影响很大，在编写完纯mpi程序后再应用OpenMP多线程并不总是最好的方案。在某些情况下，直接访问SMP上的共享内存在通信时间上的优势被这种缓存效率低下的缺点所抵消。然而，当程序自然存在一定粒度大小的负载不平衡时，混合程序的性能有可能克服具有动态负载平衡特征的MPI-only程序。垃圾邮件粒子码就是这样一种应用。

In this research, we constructed a PC cluster connecting multiple SMP-based PC nodes with various network interfaces. Such an SMP cluster system has both shared memory and distributed memory architectures together, and there are several possibility for programming like message passing, shared memory and mixture of them. Our experimental cluster contains 4-way and 2-way SMP of Pentium-III processors and two types of interconnection network are available, Myrinet800 and Fast Ethernet.We have evaluated and analyzed the performance of hybrid programming to mixture both MPI and OpenMP, and message passing programming with MPI only targeting NAS Parallel Benchmarks as basic benchmarking and SPAM (Smoothed Particle Applied Mechanics) particle code for actual scientific program. Against to the preliminary estimation, MPI-only programming achieved better performance in most of these programs. To analyze these results, we applied the cache hit ratio measurement for the research on cache behavior according to the programming style. As a result, we have cleared the hybrid program with MPI for inter-node communication and OpenMP for intra-node multithreading often breaks well-tuned cache utilization on MPI-only program.This research concludes that the performance of hybrid program is strongly affected by data access pattern, and it is not always the best scheme to apply OpenMP multithreading after we finish to write the MPI-only program. In some cases, the advantage in communication time of direct access to shared memory on SMP is overcome by the disadvantage of such cache inefficiency. However, when the program naturally has load imbalancing with a certain size of granularity, there is a possibility of the performance of hybrid program overcomes the MPI-only one with dynamic load balancing feature. SPAM particle code is one of such applications.