CSR: Small: Developing and Implementing High Performance Erasure Codes to Tolerate Storage Failures

计算机存储系统的规模正在爆炸，并且容易受到不断增加的各种故障的影响。 如果不采取措施来预测这些故障，系统将丢失数据，这将带来可怕和悲惨的后果。 保护当今和未来十年的存储系统免受可能导致数据丢失的故障的影响是至关重要的。擦除码是保护计算机存储系统在发生故障时免受数据丢失的数学实体。 使用擦除码，可以从已经存储的数据中计算和存储额外的信息，以便当磁盘出现故障或损坏时，可以从幸存者重新计算原始数据。 最佳擦除码是众所周知的小型存储系统。 然而，今天存在更大的系统，并且在未来将更加普遍，并且当前保护它们的擦除码是次优的。 该项目致力于开发、评估和实施用于大规模存储装置的改进的擦除码。该项目从目前已知的阈值开始，并探索用于更大和更复杂系统的代码。 探索是多管齐下的，解决的数学框架，结构特性和实施功能的代码。 其结果将是新的代码，性能更好，比现有的更安全。 除了代码的正式演示外，该项目还将作为开源软件库来实现和传播它们，以便需要它们的从业者可以利用这些工作，而不必掌握它们背后的复杂数学。

Computer storage systems are exploding in size and are vulnerable to an ever-increasing variety of failures. Without measures to anticipate these failures, the systems will lose data, an outcome with dire and tragic consequences. Protecting the storage systems of today and the next decade from failures that can lead to lost data is of paramount importance.Erasure codes are mathematical entitities that protect computer storage systems from losing data when they fail. With an erasure code, one computes and stores extra information from the data that is already stored, so that when disks fail or become corrupted, the original data may be recalculated from the survivors. Optimal erasure codes are well-known for small storage systems. However, larger systems exist today and will be more prevalent in the future, and the current erasure codes that protect them are suboptimal. This project attacks the unknown to develop, evaluate and implement improved erasure codes for large-scale storage installations.The project begins at the threshold of what is currently known and explores codes for larger and more complex systems. The exploration is multi-pronged, addressing the mathematical framework, structural properties and implementational features of the codes. The result will be new codes that perform better and are safer than current ones. Besides formal presentations of the codes, the project will implement and disseminate them as open-source software libraries, so that the practitioners that need them can leverage the work without having to master the complex math that underlies them.