Data Distribution for Heterogeneous Storage Systems
Data Distribution for Heterogeneous Storage Systems
复制标题
DOI：
10.1109/tc.2022.3223302
复制
发表时间：
2023-06
影响因子：
3.7
通讯作者：
Jiang Zhou;Yong Chen;Mai Zheng;Weiping Wang
中科院分区：
文献类型：
--
作者：
Jiang Zhou;Yong Chen;Mai Zheng;Weiping Wang
在许多科学和工程领域中，数据的指数级增长对存储系统提出了重大挑战。数据分发是大规模分布式存储系统中的关键组件，对于在数十万到数十万个存储设备中放置PB级及以上的数据发挥着至关重要的作用。与此同时，异构存储系统，如那些具有硬盘驱动器（HDD）和存储类存储器（SCM）的设备，由于其独特和互补的特性，已成为越来越受欢迎的海量数据存储。本文提出了一种新的数据分发算法称为SUORA（可扩展和均匀存储通过最佳自适应和随机数寻址），专门用于异构设备，以最大限度地发挥他们的优势。SUORA提供了一种完全对称、高效的方法，可以在混合和分层存储集群中分发数据。该方法将异构设备划分为不同的存储桶和存储段，采用伪随机函数将数据映射到存储桶和存储段上，均衡考虑了存储容量、性能和生命周期。通过分析热点和访问模式，SUORA逐渐将热数据从HDD移动到SCM以优化吞吐量，并将冷数据移动到SCM以实现负载平衡。它将数据复制与迁移相结合，可显著降低移动开销，同时使数据放置更适应不同的工作负载。通过仿真和Sheepdog存储系统的测试表明，SUORA充分考虑了不同设备的特性，提高了异构存储系统的整体性能。
The exponential growth of data in many science and engineering domains poses significant challenges to storage systems. Data distribution is a critical component in large-scale distributed storage systems and plays a vital role in placing petabytes of data and beyond, among tens to hundreds of thousands of storage devices. Meantime, heterogeneous storage systems, such as those having devices with hard disk drives (HDDs) and storage class memories (SCMs), have become increasingly popular for massive data storage due to their distinct and complement characteristics. This paper presents a new data distribution algorithm called SUORA (Scalable and Uniform storage via Optimally-adaptive and Random number Addressing) specifically for heterogeneous devices to maximize the benefits of them. SUORA provides a fully symmetric, highly efficient methodology to distribute data across a hybrid and tiered storage cluster. It divides heterogeneous devices into different buckets and segments, and adopts pseudo-random functions to map data onto them with the balanced consideration of capacity, performance and life-time. By analyzing hotness and access patterns, SUORA gradually moves hot data from HDDs to SCMs to optimize the throughput, and moves cold data reversely for load balance. It combines data replication with migration to significantly reduce movement overhead while making data placement more adaptive to different workloads. Extensive evaluations on simulation and Sheepdog storage system show that, with considering distinct characteristics of various devices thoroughly, SUORA improves the overall performance efficiency of heterogeneous storage systems.