A Research on Program Developing Environments of Heterogeneously Highly Parallel Programs

首先，我们构建了异构并行处理环境，该环境由四个工作站（WS）组成，其中一个工作站是具有4个处理器的SMP机器，以及通过100Mbps以太网相互连接的10台个人计算机（PC）。WSs使用SPARC处理器并运行Solaris操作系统，而pc使用各种型号的Intel x86系列处理器并运行不同类型的Linux操作系统发行版。为了验证这个并行处理环境，我们安装了PVM和MPI消息传递库，并利用这些库开发了一些并行程序。由于我们主要强调并行程序的性能方面，所以我们更喜欢使用MPI函数的消息传递范式。通过多个SPMD （Single Program, Multiple data -streams，单程序多数据流）并行程序的开发，分析了数据应该如何分布和如何优化派生的通信模式，并考察了这种编程范式的有效性和充分性。这些研究使我们相信SPMD并行编程在一定程度上是有效的，但是，中型或大型SPMD程序的生产率低得令人难以忍受，并且其可读性相当差。另一方面，我们注意到物理上分布的、逻辑上共享的内存范式的有效性，特别是OpenMP作为该范式事实上的标准的最佳候选。我们还提出了利用负载增长率及其加速度等负载信息在并行程序中多个进程之间进行动态负载平衡（和/或优化）的新方案。最后，我们开发了一个新的消息传递库，它可以降低TCP/IP协议和MPI中指定的一些通信功能的通信开销。仿真结果表明，与现有的MPI库相比，该库具有较好的性能。

First of all, we have built the heterogeneous parallel processing environment consisting of four workstations (WS), one of which is an SMP machine with 4 processors, and ten personal computers (PC) interconnected via 100Mbps Ethernet. While the WSs employ SPARC processors and run Solaris operating system (OS), the PCs employ various models of Intel x86 family processors and run different kinds of Linux OS distributions. To verify this parallel processing environment, we installed PVM and MPI message passing libraries and developed some parallel programs utilizing these libraries.Since we primarily emphasized the performance aspects of parallel programs, we prefer the message-passing paradigm using MPI functions. Through the development of several SPMD (Single Program, Multiple Data-streams) parallel programs, we analyzed how the data should be distributed and how the derived communication patterns should be optimized, and investigated the effectiveness and the adequacy of this programming paradigm. These investigations convinced us that the SPMD parallel programming is efficient to a certain extent, however, the productivity of middle or large-scale SPMD programs is intolerably low and the readability of those is rather poor. On the other hand, we noticed effectiveness of the physically distributed, logically shared memory paradigm, especially OpenMP as the best candidate of the de facto standard of this paradigm.We also proposed the novel schemes for dynamic load balancing (and/or optimizing) among many processes within a parallel program, which utilize some kinds of load information, such as the load growing rate and its acceleration. Finally, we have developed a new message passing library which can decrease the communication overheads within the TCP/IP protocol and some communication functions specified in the MPI. Simulation results showed that this new library can exhibit fairly good performance comparing to some existing MPI libraries.