Replication under scalable hashing: a family of algorithms for scalable decentralized data distribution
Replication under scalable hashing: a family of algorithms for scalable decentralized data distribution
复制标题
DOI：
10.1109/ipdps.2004.1303042
复制
发表时间：
2004-04
期刊：
影响因子：
--
通讯作者：
R. Honicky;E. L. Miller
中科院分区：
文献类型：
--
作者：
R. Honicky;E. L. Miller
仅提供摘要形式。分散式数据分发的典型算法在首次使用前完全构建的系统中效果最好;添加或删除组件会导致数据的大量重组或系统中的负载不平衡。我们已经开发了一系列分散式算法，RUSH（可扩展哈希下的复制），将复制对象映射到存储服务器或磁盘的可扩展集合。RUSH算法根据用户指定的服务器权重将对象分发到服务器。虽然所有RUSH变体都支持向系统添加服务器，但不同的变体在PB级系统中的查找时间、镜像性能（与冗余代码相反）和存储服务器删除方面具有不同的特征。所有RUSH变体在添加新服务器或删除现有服务器时重新分发尽可能少的对象，并且所有变体都保证不会将特定对象的两个副本放置在同一服务器上。由于没有中央目录，客户端可以并行计算数据位置，允许数千个客户端同时访问数千个服务器上的对象。
Summary form only given. Typical algorithms for decentralized data distribution work best in a system that is fully built before it first used; adding or removing components results in either extensive reorganization of data or load imbalance in the system. We have developed a family of decentralized algorithms, RUSH (replication under scalable hashing), that maps replicated objects to a scalable collection of storage servers or disks. RUSH algorithms distribute objects to servers according to user-specified server weighting. While all RUSH variants support addition of servers to the system, different variants have different characteristics with respect to lookup time in petabyte-scale systems, performance with mirroring (as opposed to redundancy codes), and storage server removal. All RUSH variants redistribute as few objects as possible when new servers are added or existing servers are removed, and all variants guarantee that no two replicas of a particular object are ever placed on the same server. Because there is no central directory, clients can compute data locations in parallel, allowing thousands of clients to access objects on thousands of servers simultaneously.