ePulsar: Control Plane for Publish-Subscribe Systems on Geo-Distributed Edge Infrastructure
ePulsar: Control Plane for Publish-Subscribe Systems on Geo-Distributed Edge Infrastructure
复制标题
DOI：
10.1145/3453142.3491271
复制
发表时间：
2021-12
期刊：
影响因子：
--
通讯作者：
Harshit Gupta;Tyler C. Landle;U. Ramachandran
中科院分区：
文献类型：
--
作者：
Harshit Gupta;Tyler C. Landle;U. Ramachandran
自主无人机和大型多人游戏等新兴应用需要多个地理分布式参与实体之间的实时通信。部署在地理分布式边缘基础设施上的发布-订阅系统将为此类应用程序提供可扩展的消息传递中间件。然而，由于异构的客户端代理延迟和持续的客户端移动性，Apache Pulsar 和 Kafka 等最先进的发布-订阅系统在地理分布式部署中执行效率低下。我们为地理分布式发布-订阅系统提出了一种新颖的控制平面架构，该架构能够自适应主题分区，从而为此类应用程序提供低延迟消息传递。我们利用点对点网络协调协议来可扩展地估计发布-订阅代理和客户端之间的网络延迟。客户端代理延迟和工作负载指标不断从代理收集，并用于检测延迟违规或工作负载不平衡，从而触发主题的重新分配。我们开发了 ePulsar，它将控制平面架构思想融入到流行的 Apache Pulsar 发布-订阅系统中，保留了 Pulsar 的数据平面 API。我们使用代表模拟地理分布式基础设施上典型的以边缘为中心的应用程序的工作负载场景来评估所提议的控制平面的功效和开销。
Emerging applications such as autonomous drones and massively multiplayer gaming require real-time communication between multiple geo-distributed participating entities. A publish-subscribe system deployed on a geo-distributed edge infrastructure would provide a scalable messaging middleware for such applications. However state-of-the-art publish-subscribe systems like Apache Pulsar and Kafka perform inefficiently in a geo-distributed deployment due to heterogeneous client-broker latencies and constant client mobility. We present a novel control-plane architecture for geo-distributed publish-subscribe systems that is capable of adaptive topic partitioning to enable low-latency messaging for such applications. We leverage a peer-to-peer network coordinate protocol for scalable estimation of network latencies between publish-subscribe brokers and clients. Client-broker latency and workload metrics are continuously collected from brokers and used to detect latency violations or workload imbalance, which triggers reassignment of topics. We develop ePulsar, which incorporates the control-plane architecture ideas into the popular Apache Pulsar publish-subscribe system, retaining Pulsar's data-plane APIs. We evaluate the efficacy and overheads of the proposed control plane using workload scenarios representative of typical edge-centric applications on an emulated geo-distributed infrastructure.