CSR: Small: Cost Effective, High Performance Solutions Using Erasure Codes for Big Data Management in Large Data Centers

CSR：小型：在大型数据中心使用纠删码进行大数据管理的经济高效、高性能解决方案

• 批准号: 1700719
• 负责人: Xubin He
• 金额: $ 7.11万
• 依托单位: Temple University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-08-15 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1700719&HistoricalAwards=false
• 关键词: CSR; Small; Cost; Effective; Performance

Data and I/O availability is an increasing concern in todayÕs large data centers where both data volume and complexity are increasing dramatically. Most existing solutions are based on multi-replication techniques to provide data redundancy, where data chunks are replicated across storage server nodes. However, multi-replication techniques are insufficient to manage big data: itÕs a big challenge to efficiently replicate N copies of a data set of tens-to-hundreds of petabytes! As an alternative solution, erasure codes tolerating multiple failures can provide reliability and availability at much lower cost. However, the biggest challenge using erasure codes to manage big data is the performance problem due to the complex encoding/decoding operations, which limits the application of erasure codes in large-scale data centers. This project develops cost effective techniques to exploit erasure codes to achieve high availability and enhance performance in large data centers to efficiently manage big data via several research innovations. This project cohesively investigates how to utilize proper spatial cost and system/architecture techniques to improve the overall data access performance of server clusters built upon erasure codes. This research has fundamental contributions to pave the way to efficiently deploy data centers using erasure codes. It has potential to benefit numerous big data applications such as online searching, social network, e-business, health care, and so on which are typically data intensive.

数据和I/O可用性是当今大型数据中心日益关注的问题，因为数据量和复杂性都在急剧增加。大多数现有的解决方案基于多复制技术来提供数据冗余，其中数据块跨存储服务器节点复制。然而，多复制技术不足以管理大数据：有效地复制数十到数百PB的数据集的N个副本是一个巨大的挑战！作为替代解决方案，容忍多个故障的擦除码可以以低得多的成本提供可靠性和可用性。然而，使用纠删码管理大数据的最大挑战是由于复杂的编码/解码操作导致的性能问题，这限制了纠删码在大规模数据中心中的应用。 该项目开发具有成本效益的技术，利用擦除码实现高可用性，并通过几项研究创新提高大型数据中心的性能，以有效地管理大数据。本计画将探讨如何利用适当的空间成本与系统架构技术，以改善建立在纠删码上的伺服器丛集的整体资料存取效能。这项研究为使用纠删码有效部署数据中心铺平了道路。 它有可能使许多大数据应用程序受益，例如在线搜索，社交网络，电子商务，医疗保健等，这些应用程序通常是数据密集型的。

项目成果

SELF: A High Performance and Bandwidth Efficient Approach to Exploiting Die-Stacked DRAM as Part of Memory

SELF：一种利用裸片堆叠 DRAM 作为内存一部分的高性能和带宽高效方法

• DOI: 10.1109/mascots.2017.23
• 发表时间: 2017
• 期刊: and Simulation of Computer and Telecommunication Systems (MASCOTS
• 作者: Guo, Yuhua;Liu, Qing;Xiao, Weijun;Huang, Ping;Podhorszki, Norbert;Klasky, Scott;He, Xubin
• 通讯作者: He, Xubin