A Research on Program Developing Environments of Heterogeneously Highly Parallel Programs

首先，我们搭建了一个由4台工作站(WS)和10台个人计算机(PC)通过100 Mbps以太网互连而成的异构型并行处理环境。虽然WSS使用SPARC处理器并运行Solaris操作系统(OS)，但PC使用各种型号的Intel x86系列处理器并运行不同类型的Linux OS发行版。为了验证这种并行处理环境，我们安装了PVM和MPI消息传递库，并利用这些库开发了一些并行程序，由于我们主要强调并行程序的性能方面，所以我们倾向于使用MPI函数进行消息传递。通过几个SPMD(Single Program，多数据流)并行程序的开发，分析了数据应该如何分布以及派生的通信模式应该如何优化，并研究了这种编程范型的有效性和充分性。这些研究使我们相信SPMD并行编程在一定程度上是有效的，然而，中、大型SPMD程序的生产率低得令人无法忍受，可读性相当差。另一方面，我们注意到物理分布的、逻辑共享的存储范例的有效性，特别是OpenMP作为该范例事实上的标准的最佳候选者，我们还提出了在并行程序内多个进程之间的动态负载平衡(和/或优化)的新方案，该方案利用诸如负载增长率及其加速等负载信息。最后，我们开发了一个新的消息传递库，它可以减少TCP/IP协议中的通信开销和MPI中规定的一些通信功能。仿真结果表明，与现有的MPI库相比，新的MPI库具有较好的性能。

First of all, we have built the heterogeneous parallel processing environment consisting of four workstations (WS), one of which is an SMP machine with 4 processors, and ten personal computers (PC) interconnected via 100Mbps Ethernet. While the WSs employ SPARC processors and run Solaris operating system (OS), the PCs employ various models of Intel x86 family processors and run different kinds of Linux OS distributions. To verify this parallel processing environment, we installed PVM and MPI message passing libraries and developed some parallel programs utilizing these libraries.Since we primarily emphasized the performance aspects of parallel programs, we prefer the message-passing paradigm using MPI functions. Through the development of several SPMD (Single Program, Multiple Data-streams) parallel programs, we analyzed how the data should be distributed and how the derived communication patterns should be optimized, and investigated the effectiveness and the adequacy of this programming paradigm. These investigations convinced us that the SPMD parallel programming is efficient to a certain extent, however, the productivity of middle or large-scale SPMD programs is intolerably low and the readability of those is rather poor. On the other hand, we noticed effectiveness of the physically distributed, logically shared memory paradigm, especially OpenMP as the best candidate of the de facto standard of this paradigm.We also proposed the novel schemes for dynamic load balancing (and/or optimizing) among many processes within a parallel program, which utilize some kinds of load information, such as the load growing rate and its acceleration. Finally, we have developed a new message passing library which can decrease the communication overheads within the TCP/IP protocol and some communication functions specified in the MPI. Simulation results showed that this new library can exhibit fairly good performance comparing to some existing MPI libraries.