Joint Latency and Cost Optimization for Erasure-Coded Data Center Storage
Joint Latency and Cost Optimization for Erasure-Coded Data Center Storage
复制标题
DOI：
10.1109/tnet.2015.2466453
复制
发表时间：
2014-04
期刊：
影响因子：
--
通讯作者：
Yu Xiang;Tian Lan;V. Aggarwal;Y. Chen
中科院分区：
文献类型：
--
作者：
Yu Xiang;Tian Lan;V. Aggarwal;Y. Chen
现代的分布式存储系统提供了很大的能力，可以满足成倍增长的存储空间需求。他们经常使用擦除代码来防止磁盘和节点失败以提高可靠性，同时试图满足应用程序和客户的延迟要求。本文提供了具有任意服务时间分布的这种擦除编码存储的平均服务延迟，由多个异构文件组成。结果不仅取代仅适用于单个文件或均匀文件的已知延迟界限，还可以在三个维度上最小化的联合延迟和存储成本最小化的新问题：选择擦除代码，安装编码块的放置以及优化日程安排政策。通过计算具有可证明的收敛性的凸近近似值来有效解决该问题。我们在三个地理分布的数据中心的开源云存储部署中进一步原型解决方案。实验结果验证了我们的理论延迟分析并显示出明显的延迟减少，从而为擦除编码的存储中提出的延迟成本折衷提供了宝贵的见解。
Modern distributed storage systems offer large capacity to satisfy the exponentially increasing need of storage space. They often use erasure codes to protect against disk and node failures to increase reliability, while trying to meet the latency requirements of the applications and clients. This paper provides an insightful upper bound on the average service delay of such erasure-coded storage with arbitrary service time distribution and consisting of multiple heterogeneous files. Not only does the result supersede known delay bounds that only work for a single file or homogeneous files, it also enables a novel problem of joint latency and storage cost minimization over three dimensions: selecting the erasure code, placement of encoded chunks, and optimizing scheduling policy. The problem is efficiently solved via the computation of a sequence of convex approximations with provable convergence. We further prototype our solution in an open-source cloud storage deployment over three geographically distributed data centers. Experimental results validate our theoretical delay analysis and show significant latency reduction, providing valuable insights into the proposed latency-cost tradeoff in erasure-coded storage.