Demand-Aware Erasure Coding for Distributed Storage Systems
Demand-Aware Erasure Coding for Distributed Storage Systems
复制标题
分布式存储系统的需求感知擦除编码
DOI：
--
复制
发表时间：
2018
影响因子：
6.5
通讯作者：
Baochun Li
中科院分区：
文献类型：
--
作者：
Jun Yu Li;Baochun Li
分布式存储系统通过将数据存储在商用存储服务器上来提供云存储服务。传统上，通过复制来保护数据免受这种商品服务器的故障。擦除编码比复制消耗更少的存储开销来容忍相同数量的故障，因此在许多分布式存储系统中已经取代了复制。然而，使用纠删编码，从故障重构数据的开销也显著增加。在数据访问高度偏斜的不断变化的工作负载下，部署具有适当参数值的纠删编码以实现存储开销和重构开销之间的良好权衡是具有挑战性的。在本文中，我们提出了斑马，一个框架，编码数据的需求到多个层次，部署具有不同的参数值的纠删码。Zebra自动确定这些层的数量，并将具有最佳参数值的擦除码动态分配到相应的层中。使用Zebra，可以通过多层实现存储开销和重建开销之间的灵活权衡。当需求发生变化时，Zebra会通过边际网络传输量进行自我调整。我们证明，斑马可以与两个代表性的家庭的纠删码在分布式存储系统中，里德-所罗门码和本地重建码。
Distributed storage systems provide cloud storage services by storing data on commodity storage servers. Conventionally, data are protected against failures of such commodity servers by replication. Erasure coding consumes less storage overhead than replication to tolerate the same number of failures and thus has been replacing replication in many distributed storage systems. However, with erasure coding, the overhead of reconstructing data from failures also increases significantly. Under the ever-changing workload where data accesses can be highly skewed, it is challenging to deploy erasure coding with appropriate values of parameters to achieve a well trade-off between storage overhead and reconstruction overhead. In this paper, we propose Zebra, a framework that encodes data by their demand into multiple tiers that deploy erasure codes with different values of parameters. Zebra automatically determines the number of such tiers and dynamically assigns erasure codes with optimal values of parameters into corresponding tiers. With Zebra, a flexible trade-off between storage overhead and reconstruction overhead is achieved with multiple tiers. When demand changes, Zebra adjusts itself with a marginal amount of network transfer. We demonstrate that Zebra can work with two representative families of erasure codes in distributed storage systems, Reed-Solomon codes and local reconstruction codes.