CSR: Small: Developing and Implementing High Performance Erasure Codes to Tolerate Storage Failures

计算机存储系统的大小呈爆炸式增长，并且容易受到越来越多的各种故障的影响。如果不采取措施预测这些故障，系统将丢失数据，这将带来可怕而悲惨的后果。保护当今和未来十年的存储系统免受可能导致数据丢失的故障的影响是至关重要的。擦除码是一种数学实体，用于保护计算机存储系统在发生故障时不会丢失数据。使用擦除码，可以从已存储的数据计算和存储额外信息，以便当磁盘出现故障或损坏时，可以从幸存的磁盘重新计算原始数据。最优擦除码对于小型存储系统是众所周知的。然而，更大的系统今天存在，并将在未来更加普遍，目前保护它们的擦除码不是最优的。该项目旨在开发、评估和实现用于大规模存储设备的改进的擦除码。该项目从目前已知的基础开始，探索用于更大和更复杂系统的代码。探索是多管齐下的，涉及代码的数学框架、结构属性和实现特征。其结果将是比现有代码执行得更好、更安全的新代码。除了正式介绍这些代码外，该项目还将以开源软件库的形式实施和传播它们，以便需要它们的从业者可以利用这些工作，而不必掌握它们背后的复杂数学。

Computer storage systems are exploding in size and are vulnerable to an ever-increasing variety of failures. Without measures to anticipate these failures, the systems will lose data, an outcome with dire and tragic consequences. Protecting the storage systems of today and the next decade from failures that can lead to lost data is of paramount importance.Erasure codes are mathematical entitities that protect computer storage systems from losing data when they fail. With an erasure code, one computes and stores extra information from the data that is already stored, so that when disks fail or become corrupted, the original data may be recalculated from the survivors. Optimal erasure codes are well-known for small storage systems. However, larger systems exist today and will be more prevalent in the future, and the current erasure codes that protect them are suboptimal. This project attacks the unknown to develop, evaluate and implement improved erasure codes for large-scale storage installations.The project begins at the threshold of what is currently known and explores codes for larger and more complex systems. The exploration is multi-pronged, addressing the mathematical framework, structural properties and implementational features of the codes. The result will be new codes that perform better and are safer than current ones. Besides formal presentations of the codes, the project will implement and disseminate them as open-source software libraries, so that the practitioners that need them can leverage the work without having to master the complex math that underlies them.