CPA-ACR: Fast Recovery Using Optimal and Near-Optimal Parallelism in Data-Intensive Computing

点击翻译按钮获取中文摘要

Recent years have seen the emergence of large-scale data clusters consisting of hundreds of thousands hard drives in data-intensive computing. Because failure of a disk could cause loss of valuable data or information, the direct and indirect costs of ever often disk failures are becoming increasingly critical issues in the deployment and operation of these computational platforms. Recent studies conclude that the trend in data-intensive computing is towards much higher disk replacement rates in the field than the Mean-Time-To-Failure estimates of the manufacturer datasheet would suggest, even in the first years of operation. Hence, the recovery becomes state of storage. Existing storage recovery solutions successfully developed optimal and near-optimal parallelism layouts such as declustered parity organizations at small-scale storage architectures. There are very few studies on multi-way replication based storage architectures that are equally important but significantly different from erasure code based storage architectures. Moreover, it is difficult to scale up to a large size because current placement-ideal solutions have a limited number of configurations. Lastly, fast recovery demands efficient reverse data lookup, which is not well studied in current scalable data distribution schemes. The investigators develop methods and tools for achieving fast recovery by exploiting optimal and near-optimal parallelism techniques, and distributed hash table and reverse hashing techniques to improve the scalability of reverse data lookup in high-performance storage systems. The proposed research, if successful, will have broad impact in both fault-tolerance computing and high-performance computing community by providing a scalable and fast storage recovery solution.