Repair Pipelining for Erasure-coded Storage: Algorithms and Evaluation
Repair Pipelining for Erasure-coded Storage: Algorithms and Evaluation
复制标题
纠删码存储的修复流水线：算法和评估
DOI：
10.1145/3436890
复制
发表时间：
2021-06-01
影响因子：
1.7
通讯作者：
Hu, Yuchong
中科院分区：
文献类型：
--
作者：
Li, Xiaolu;Yang, Zuoru;Hu, Yuchong
我们提出了修复流水线，一种技术，加快修复性能一般擦除编码存储。通过以流水线方式跨存储节点以小尺寸单元仔细调度故障数据的修复，修复流水线将单个块修复时间减少到与同构环境中单个块的正常读取时间大致相同。我们进一步设计了不同的扩展修复流水线算法的异构环境和多块修复操作。我们实现了一个修复流水线原型，称为ECPipe，并将其作为一个中间件系统集成到两个版本的Hadoop分布式文件系统（HDFS）（即HDFS-RAID和HDFS-3）以及Quantcast文件系统。在本地测试平台和Amazon EC2上进行的实验表明，修复流水线与现有修复技术相比，显著提高了降级读取和全节点恢复的性能。
We propose repair pipelining, a technique that speeds up the repair performance in general erasure-coded storage. By carefully scheduling the repair of failed data in small-size units across storage nodes in a pipelined manner, repair pipelining reduces the single-block repair time to approximately the same as the normal read time for a single block in homogeneous environments. We further design different extensions of repair pipelining algorithms for heterogeneous environments and multi-block repair operations. We implement a repair pipelining prototype, called ECPipe, and integrate it as a middleware system into two versions of Hadoop Distributed File System (HDFS) (namely, HDFS-RAID and HDFS-3) as well as Quantcast File System. Experiments on a local testbed and Amazon EC2 show that repair pipelining significantly improves the performance of degraded reads and full-node recovery over existing repair techniques.