A research on interconnection network for large scale clusters based on commodity network

在本研究中，我们建立了一种技术，为PC集群提供高性能，可扩展和可靠的互连网络，基于商用局域网，如千兆以太网，利用它们的多个链路。在我们的技术中，我们仅通过软件实现所有这些功能。该系统由两个部分组成；(1)利用多个网络链路将两者结合起来，实现高带宽和冗余；(2)克服传统以太网利用中性能可扩展性的限制，采用简单的树型结构，通过附加的以太网驱动程序控制标准标记vlan技术，提供多路径解决方案。这两种技术分别被命名为RI2N/UDP和VFREC-Net。我们构建了一个有16个节点的PC集群测试平台系统，每个节点配备双千兆以太网链路来开发系统并评估性能和功能。因此，RI2N/UDP提供了双倍的带宽和双链路，效率在95%以上。当两条链路中的一条发生故障时，通信流量自动重定向到另一条链路，以提供用户透明的容错功能。在这种情况下，单链路系统也可以提供95%以上的效率。在vfrecc - net的评估中，我们可以用普通的廉价的第二层以太网交换机和我们的软件构建一个多级FAT-Tree配置。为了进行可扩展性测试，我们将软件移植到另一个64节点的平台上，并证实了具有4条上行链路的FAT-Tree正好提供了4倍的带宽，能够扩展到数千个PC集群节点。最后，在我们研究的最后阶段，我们可以成功地将两种技术结合起来，为PC集群提供高带宽，可靠和可扩展的互连解决方案，以实现高性能计算。这三个特性对于大规模高性能计算来说是必不可少的，我们在本研究中证明了仅使用低价格的千兆以太网和由开发的驱动程序和通信库控制的第二层交换机是可能的。少

In this research, we have established a technology to provide high-performance, scalable and reliable interconnection network for PC clusters, based on commodity LAN such as Gigabit Ethernet to utilize multiple links of them. In our technology, we implemented all these functionalities by software only. The system consists of two components ; (1) utilizing multiple network links to combine both for high-bandwidth and redundancy, and (2) conquering the limitation of performance scalability on traditional Ethernet utilization, a simple tree construction, to provide multi-path solution with standard tagged-VLAN technology controlled by additional Ethernet driver. These two technologies are named as RI2N/UDP and VFREC-Net, respectively.We constructed a test-bed PC cluster system with 16 nodes where each node is equipped with dual Gigabit Ethernet links to develop the system and evaluate the performance and functionality. As a result, RI2N/UDP provides doubled bandwidth with dual links with … More more than 95% of efficiency. When one of two links fails, the communication traffic is automatically redirected to the other link to provide user-transparent fault tolerance. In this situation, the system with a single link also provides more than 95% efficiency. In the evaluation of VFREC-Net, we could construct a multi-stage FAT-Tree configuration with ordinary and inexpensive Layer-2 Ethernet switch and our software. For the scalability test, we ported the software to another platform with 64 nodes, and confirmed the FAT-Tree with four up-links exactly provides four times bandwidth which is enable to expand for thousands of PC cluster nodes.Finally at the last stage of our research, we could successfully combined two technologies to provide high-bandwidth, reliable and scalable interconnection solution for cost/effective PC cluster for high performance computing. These three features are really essential for large scale HPC, and we proved that it is possible with only low-price Gigabit Ethernet and Layer-2 switches controlled by the developed drivers and communication libraries in this research. Less