Beyond MPI: New Communication Interfaces for Database Systems and Data-Intensive Applications
Beyond MPI: New Communication Interfaces for Database Systems and Data-Intensive Applications
复制标题
超越 MPI：数据库系统和数据密集型应用程序的新通信接口
DOI：
10.1145/3456859.3456862
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Swart, Garret
中科院分区：
文献类型：
--
作者：
Liu, Feilong;Barthels, Claude;Blanas, Spyros;Kimura, Hideaki;Swart, Garret
支持远程DirectMemoryAccess（RDMA）的网络正变得越来越普遍。然而，RDMA为远程存储器提供了一个有限的编程接口，包括读、写和原子操作。仅使用RDMA，完成对远程数据结构的最基本操作通常需要通过网络进行多次往返。数据密集型系统强烈需要更高级别的通信抽象，以支持更复杂的交互模式。一个自然的候选者是MPI，它是HPC社区中开发高性能应用程序的事实上的标准。本文批判性地评估了MPI的通信原语，并表明在数据处理系统的背景下使用MPI带来了一系列无法克服的挑战。基于这种分析，我们提出了一个新的通信抽象RDMO，或远程DirectMemory操作，调度一个短序列的读取，写入和原子操作到远程内存，并在一个单一的往返执行它们。
Networkswith Remote DirectMemoryAccess (RDMA) support are becoming increasingly common. RDMA, however, offers a limited programming interface to remote memory that consists of read, write and atomic operations. With RDMA alone, completing the most basic operations on remote data structures often requires multiple round-trips over the network. Data-intensive systems strongly desire higher-level communication abstractions that supportmore complex interaction patterns.A natural candidate to consider is MPI, the de facto standard for developing high-performance applications in the HPC community. This paper critically evaluates the communication primitives of MPI and shows that using MPI in the context of a data processing system comes with its own set of insurmountable challenges. Based on this analysis, we propose a new communication abstraction named RDMO, or Remote DirectMemory Operation, that dispatches a short sequence of reads, writes and atomic operations to remote memory and executes them in a single round-trip.