Performance Evaluation of the Impact of NUMA on One-sided RDMA Interactions
Performance Evaluation of the Impact of NUMA on One-sided RDMA Interactions
复制标题
DOI：
10.1109/srds51746.2020.00036
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
J. Nelson;R. Palmieri
中科院分区：
文献类型：
--
作者：
J. Nelson;R. Palmieri
远程直接存储器访问（RDMA）和非统一存储器访问（NUMA）是现代高性能计算平台的关键技术。RDMA允许节点直接访问远程机器上的内存。多处理器体系结构实现NUMA以提高内存访问性能。当这些技术组合在一起时，在某些配置下会表现出性能损失。本文是第一个研究，探讨这些配置，以提供定量的结果NUMA的影响，基于RDMA的系统。超快网络的后果之一是，NUMA局部性的已知含义现在对支持RDMA的分布式系统的性能构成了更高的相对影响。我们的研究量化了它的作用，并揭示了意想不到的行为。总之，远程可访问内存的NUMA局部性差会导致性能自动下降20%。此外，在远程可访问内存上运行的本地工作负载可能导致300%的性能差距，具体取决于内存位置。令人惊讶的是，配置证明这一结果相矛盾的假设的影响NUMA的地方。我们的研究结果验证使用两代的RDMA卡，合成基准，和流行的应用程序Memcached的RDMA移植。
Remote direct memory access (RDMA) and non-uniform memory access (NUMA) are critical technologies of modern high-performance computing platforms. RDMA allows nodes to directly access memory on remote machines. Multiprocessor architectures implement NUMA to scale up memory access performance. When paired together, these technologies exhibit performance penalties under certain configurations. This paper is the first study to explore these configurations to provide quantitative findings on the impact of NUMA for RDMA-based systems. One of the consequences of ultra-fast networks is that known implications of NUMA locality now constitute a higher relative impact on the performance of RDMA-enabled distributed systems. Our study quantifies its role and uncovers unexpected behavior. In summary, poor NUMA locality of remotely accessible memory can lead to an automatic 20% performance degradation. Additionally, local workloads operating on remotely accessible memory can lead to 300% performance gap depending on memory locality. Surprisingly, configurations demonstrating this result contradict the presumed impact of NUMA locality. Our findings are validated using two generations of RDMA cards, a synthetic benchmark, and the popular application Memcached ported for RDMA.