Performance evaluation of InfiniBand with PCI Express
Performance evaluation of InfiniBand with PCI Express
复制标题
InfiniBand 与 PCI Express 的性能评估
DOI：
--
复制
发表时间：
2004
期刊：
影响因子：
--
通讯作者：
D. Panda
中科院分区：
文献类型：
--
作者：
Jiuxing Liu;Amith R. Mamidala;Abhinav Vishnu;D. Panda
我们提出了一个初步的性能评估InfiniBand HCA（主机通道适配器）从Mellanox与PCI Express接口。我们比较了使用PCI-X接口的HCA的性能。我们的研究结果表明，采用PCI Express的InfiniBand HCA可以实现显著的性能优势。与使用64位/133 MHz PCI-X接口的HCA相比，它们可以实现小消息延迟降低20%-30%。PCI Express实现的小消息延迟约为3.8 /spl mu/s，而PCI-X为5.0 /spl mu/s。对于大型消息，使用单个端口的PCI Express HCA可以提供高达968 MB/s的单向带宽和高达1916 MB/s的双向带宽，分别是使用PCI-X的HCA所实现的峰值带宽的1.24和2.02倍。当HCA的两个端口都被激活时，具有PCI Express的HCA可以提供1486 MB/s的峰值单向带宽和高达2729 MB/s的聚合双向带宽，这是使用具有PCI-X的HCA获得的峰值带宽的1.93和2.88倍。PCI Express还提高了MPI级别的性能。对于小消息，PCI Express实现了4.6 /spl mu/s的延迟。对于大消息，观察到1497 MB/s的单向带宽和2724 MB/s的双向带宽。
We present an initial performance evaluation of InfiniBand HCAs (host channel adapters) from Mellanox with PCI Express interfaces. We compare the performance with HCAs using PCI-X interfaces. Our results show that InfiniBand HCAs with PCI Express can achieve significant performance benefits. Compared with HCAs using 64 bit/133 MHz PCI-X interfaces, they can achieve 20%-30% lower latency for small messages. The small message latency achieved with PCI Express is around 3.8 /spl mu/s, compared with the 5.0 /spl mu/s with PCI-X. For large messages, HCAs with PCI Express using a single port can deliver unidirectional bandwidth up to 968 MB/s and bidirectional bandwidth up to 1916 MB/s, which are, respectively, 1.24 and 2.02 times the peak bandwidths achieved by HCAs with PCI-X. When both the ports of the HCAs are activated, HCAs with PCI Express can deliver a peak unidirectional bandwidth of 1486 MB/s and aggregate bidirectional bandwidth up to 2729 MB/s, which are 1.93 and 2.88 times the peak bandwidths obtained using HCAs with PCI-X. PCI Express also improves performance at the MPI level. A latency of 4.6 /spl mu/s with PCI Express is achieved for small messages. And for large messages, unidirectional bandwidth of 1497 MB/s and bidirectional bandwidth of 2724 MB/s are observed.