Smash: Flexible, Fast, and Resource-efficient Placement and Lookup of Distributed Storage
Smash: Flexible, Fast, and Resource-efficient Placement and Lookup of Distributed Storage
复制标题
DOI：
10.1145/3589977
复制
发表时间：
2023-05
期刊：
影响因子：
--
通讯作者：
Yi Liu;Shouqian Shi;Minghao Xie;Heiner Litz;Chen Qian
中科院分区：
文献类型：
--
作者：
Yi Liu;Shouqian Shi;Minghao Xie;Heiner Litz;Chen Qian
大规模的分布式存储系统（例如对象存储）通常应用基于哈希的位置和查找方法来达到可扩展性和资源效率。但是，当对象位置由哈希值确定时，放置变得僵化，无法优化或满足应用程序要求，例如负载平衡，失败公差，并行性和网络/系统性能。这项工作提出了一种新的解决方案，以实现两个世界的最佳状态：灵活性，同时保持成本效益和可扩展性。提出的方法Smash是一种对象放置和查找方法，可实现完整的放置灵活性，平衡负载，资源低和延迟。 Smash利用了最近的空格数据结构，并将其应用于对象位置查找。我们将Smash作为原型系统实施，并在公共云中进行评估。分析和实验结果表明，与现有基于哈希的解决方案（例如CEPH和MAPX）相比，SMASH可实现完整的放置灵活性，快速存储操作，从节点动力学快速恢复（<60％）（<60％）。
Large-scale distributed storage systems, such as object stores, usually apply hashing-based placement and lookup methods to achieve scalability and resource efficiency. However, when object locations are determined by hash values, placement becomes inflexible, failing to optimize or satisfy application requirements such as load balance, failure tolerance, parallelism, and network/system performance. This work presents a novel solution to achieve the best of two worlds: flexibility while maintaining cost-effectiveness and scalability. The proposed method Smash is an object placement and lookup method that achieves full placement flexibility, balanced load, low resource cost, and short latency. Smash utilizes a recent space-efficient data structure and applies it to object-location lookups. We implement Smash as a prototype system and evaluate it in a public cloud. The analysis and experimental results show that Smash achieves full placement flexibility, fast storage operations, fast recovery from node dynamics, and lower DRAM cost (<60%) compared to existing hash-based solutions such as Ceph and MapX.