Network Coding for Distributed Storage Systems
Network Coding for Distributed Storage Systems
复制标题
DOI：
10.1109/infcom.2007.232
复制
发表时间：
2007-02
期刊：
影响因子：
--
通讯作者：
A. Dimakis;Brighten Godfrey;M. Wainwright;K. Ramchandran
中科院分区：
文献类型：
--
作者：
A. Dimakis;Brighten Godfrey;M. Wainwright;K. Ramchandran
对等分布式存储系统通过互联网上节点上的冗余来提供对数据的可靠访问。一个关键目标是最大限度地减少用于维持冗余的带宽量。使用擦除码将文件存储在跨节点的片段中，与简单复制相比，需要更少的冗余，因此需要更少的维护带宽来提供相同的可靠性。然而，由于碎片必须定期更换节点故障，一个关键的问题是如何生成一个新的碎片在分布式的方式，同时在网络上传输尽可能少的数据。在本文中，我们介绍了一种通用的技术来分析存储体系结构，联合收割机结合任何形式的编码和复制，以及提出了两个新的计划，保持冗余使用纠删码。首先，我们展示了如何直接从系统中现有的片段优化生成MDS片段。其次，我们引入了一个新的计划，称为再生码，使用稍大的片段比MDS，但具有较低的整体带宽使用。我们还表明，通过模拟，在现实的环境中，再生代码可以减少维护带宽的使用25%或以上，与以前最好的设计-复制和擦除码的混合-同时简化系统架构。
Peer-to-peer distributed storage systems provide reliable access to data through redundancy spread over nodes across the Internet. A key goal is to minimize the amount of bandwidth used to maintain that redundancy. Storing a file using an erasure code, in fragments spread across nodes, promises to require less redundancy and hence less maintenance bandwidth than simple replication to provide the same level of reliability. However, since fragments must be periodically replaced as nodes fail, a key question is how to generate a new fragment in a distributed way while transferring as little data as possible across the network. In this paper, we introduce a general technique to analyze storage architectures that combine any form of coding and replication, as well as presenting two new schemes for maintaining redundancy using erasure codes. First, we show how to optimally generate MDS fragments directly from existing fragments in the system. Second, we introduce a new scheme called regenerating codes which use slightly larger fragments than MDS but have lower overall bandwidth use. We also show through simulation that in realistic environments, regenerating codes can reduce maintenance bandwidth use by 25% or more compared with the best previous design - a hybrid of replication and erasure codes - while simplifying system architecture.