EC-Fusion: An Efficient Hybrid Erasure Coding Framework to Improve Both Application and Recovery Performance in Cloud Storage Systems
EC-Fusion: An Efficient Hybrid Erasure Coding Framework to Improve Both Application and Recovery Performance in Cloud Storage Systems
复制标题
DOI：
10.1109/ipdps47924.2020.00029
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Han Qiu;Chentao Wu;Jie Li;M. Guo;Tong Liu;Xubin He;Yuanyuan Dong;Yafei Zhao
中科院分区：
文献类型：
--
作者：
Han Qiu;Chentao Wu;Jie Li;M. Guo;Tong Liu;Xubin He;Yuanyuan Dong;Yafei Zhao
如今，擦除编码是云存储系统中最重要的技术之一，在这些系统中，在大量数据访问中提供了快速的平行I/O处理和高功能。配置，由几种经典的擦除代码（例如Reed-Solomon（RS）代码，本地重建代码（LRC），最小存储再生（MSR）代码）支持的配置，等待在线恢复过程，前景申请工作负载和背景恢复工作负载是同时处理的，这需要对两种类型的工作负载特征进行全面的了解。过程中，它们通常是单方面的，因为以下事实是，以上两个工作量没有合并在一起以实现高成本有效的性能。要解决此问题，我们提出了擦除代码（EC融合），这是云存储系统中有效的混合擦除编码框架。恢复工作负载中的数据丢失或低风险，EC融合使用RS代码来降低计算开销和存储成本。在工作负载中，MSR代码是一个适当的选择与传统的混合擦除技术相比，EC融合将应用的响应时间升高高达1.77倍，并将重建时间降低高达69.10％。
Nowadays erasure coding is one of the most significant techniques in cloud storage systems, which provides both quick parallel I/O processing and high capabilities of fault tolerance on massive data accesses. In these systems, triple disk failure tolerant arrays (3DFTs) is a typical configuration, which is supported by several classic erasure codes like Reed-Solomon (RS) codes, Local Reconstruction Codes (LRC), Minimum Storage Regeneration (MSR) codes, etc. For an online recovery process, the foreground application workloads and the background recovery workloads are handled simultaneously, which requires a comprehensive understanding on both two types of workload characteristics. Although several techniques have been proposed to accelerate the I/O requests of online recovery processes, they are typically unilateral due to the fact that the above two workloads are not combined together to achieve high cost-effective performance.To address this problem, we propose Erasure Codes Fusion (EC-Fusion), an efficient hybrid erasure coding framework in cloud storage systems. EC-Fusion is a combination of RS and MSR codes, which dynamically selects the appropriate code based on its properties. On one hand, for write-intensive application workloads or low risk on data loss in recovery workloads, EC-Fusion uses RS code to decrease the computational overhead and storage cost concurrently. On the other hand, for read-intensive or frequent reconstruction in workloads, MSR code is a proper choice. Therefore, a better overall application and recovery performance can be achieved in a cost-effective fashion. To demonstrate the effectiveness of EC-Fusion, several experiments are conducted in hadoop systems. The results show that, compared with the traditional hybrid erasure coding techniques, EC-Fusion accelerates the response time for application by up to 1.77×, and reduces the reconstruction time by up to 69.10%.