Pegasus: Tolerating Skewed Workloads in Distributed Storage with In-Network Coherence Directories
Pegasus: Tolerating Skewed Workloads in Distributed Storage with In-Network Coherence Directories
复制标题
DOI：
--
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
Jialin Li;J. Nelson;Ellis Michael;Xin Jin;Dan R. K. Ports
中科院分区：
文献类型：
--
作者：
Jialin Li;J. Nelson;Ellis Michael;Xin Jin;Dan R. K. Ports
高性能的分布式存储系统面临着负载不平衡的挑战。本文介绍了Pegasus，这是一种新的存储系统，它利用新一代可编程交换机ASIC来平衡存储服务器之间的负载。Pegasus使用数据存储中最流行对象的选择性复制来分配负载。Pegasus交换机使用一种新颖的网络内一致性目录，跟踪和管理复制对象的位置。这使它能够实现负载感知转发和复制密钥的动态重新平衡，同时仍然保证数据的一致性和一致性。Pegasus的设计是实用的，因为它只在交换机数据平面中存储转发元数据。由此产生的系统在延迟SLO下将分布式内存中键值存储的吞吐量提高了10倍以上-结果在具有不同程度的偏斜、读/写比率、对象大小和动态性的大量工作负载中保持不变。
High performance distributed storage systems face the challenge of load imbalance caused by skewed and dynamic workloads. This paper introduces Pegasus, a new storage system that leverages new-generation programmable switch ASICs to balance load across storage servers. Pegasus uses selective replication of the most popular objects in the data store to distribute load. Using a novel in-network coherence directory, the Pegasus switch tracks and manages the location of replicated objects. This allows it to achieve load-aware forwarding and dynamic rebalancing for replicated keys, while still guaranteeing data coherence and consistency. The Pegasus design is practical to implement as it stores only forwarding metadata in the switch data plane. The resulting system improves the throughput of a distributed in-memory key-value store by more than 10x under a latency SLO — results which hold across a large set of workloads with varying degrees of skew, read/write ratio, object sizes, and dynamism.