Studies on Remote Parallel Pipe Mechanism for Wide-Area Distributed Metacomputing

点击翻译按钮获取中文摘要

In wide-area distributed metacomputing by virtually integrated multiple computing resources on WAN, sufficient performance cannot be achieved through conventional LAN-oriented parallel processing methods based on the data-parallel approach. Therefore, this project focused on remote parallel pipe mechanism which performs high-speed parallel data transfer from one parallel program to another parallel program on a remote site like a pipeline, to alleviate the negative effect of the communication latency.In 2003,a prototype system successfully achieved parallel data transfer between multiple MPI programs running on distinct parallel machines. Since this mechanism is not incorporated in the standard MPI functions, it is an original contribution of this project. Network striping technique was also adopted in the prototype to fully exploit the communication band width between parallel computers, and its effectiveness was verified through experiments.In 2004,this prototype was upgraded to an integrated function library which can be used on generic PC clusters. In a typical PC cluster, external communication link is limited to only one node to save the number of global IP addresses assigned, and the parallel pipe approach faces a difficulty in data scattering/gathering and network striping control. The completed library can hide this complicated work from the programmer's view. Other research results are related to IPv6 technology to utilize other nodes having no external link, and authentication mechanism to achieve safe wide-area distributed metacomputing.