CIF: Small: Collaborative Research: Efficient Codes and their Performance Limits for Distributed Storage Systems

CIF：小型：协作研究：分布式存储系统的高效代码及其性能限制

• 批准号: 1422955
• 负责人: Alexander Barg
• 金额: $ 29.95万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1422955&HistoricalAwards=false
• 关键词: CIF; Small; Collaborative; Research; Efficient

The explosive growth of data being generated and collected has rekindled interest in efficient means of storing such data. Large data centers and distributed storage systems have become more widespread, playing an ever-increasing role in our everyday computational tasks. While a data center should never lose data, disk failures occur on a daily basis as confirmed by the industry statistics. Methods and ideas from error correcting codes developed in this project enable the system to provide better guarantees against data loss as well as to reduce the amount of data that needs to be moved in order to enable recovery of information lost due to disk failures. Another related goal of this project is the reduction of storage overhead needed to support the recovery procedures. These goals are accomplished by relying on algebraic methods of constructing the data encoding procedures as well as on novel algorithms of data exchange and recovery. Overall the research performed in the course of this project contributes to the development of more efficient data management procedures in large-scale distributed storage systems.This project puts forward new algebraic procedures for data encoding and recovery that enables one to achieve tradeoff between overhead and repair bandwidth based on the concept of local recovery. The project studies both the case of recovering from a single disk loss, which is the most frequent problem in systems, as well as from the failure of multiple disks, addressing the problem of correcting one erasure as well as multiple erasures in data encoding. New bounds on the distance of codes with the locality requirement derived in this research are attained with new constructions of optimal locally recoverable codes equipped with simple recovery procedures. The project also addresses the problem of simultaneous recovery of data from multiple locations, enhancing data availability in large-scale distributed storage systems which are a key backbone component of the 21st century economy.

正在生成和收集的数据的爆炸性增长重新点燃了对存储这种数据的有效手段的兴趣。 大型数据中心和分布式存储系统已经变得越来越普遍，在我们的日常计算任务中发挥着越来越重要的作用。 虽然数据中心永远不会丢失数据，但磁盘故障每天都会发生，这一点已被行业统计数据所证实。在这个项目中开发的纠错码的方法和思想使系统能够提供更好的保证，防止数据丢失，以及减少需要移动的数据量，以便恢复由于磁盘故障而丢失的信息。该项目的另一个相关目标是减少支持恢复过程所需的存储开销。这些目标是通过依赖于构建数据编码过程的代数方法以及数据交换和恢复的新型算法来实现的。总的来说，在这个项目的过程中进行的研究有助于更有效的数据管理程序在大规模的分布式存储systems.This项目的发展提出了新的代数程序的数据编码和恢复，使一个实现之间的权衡开销和修复带宽的基础上的本地恢复的概念。 该项目研究了从单个磁盘丢失中恢复的情况，这是系统中最常见的问题，以及从多个磁盘故障中恢复的情况，解决了纠正数据编码中的一个擦除和多个擦除的问题。新的边界上的距离的代码与本地性的要求，在这项研究中获得了新的建设最佳的本地可恢复的代码配备简单的恢复程序。该项目还解决了从多个位置同时恢复数据的问题，提高了作为21世纪世纪经济关键骨干组成部分的大规模分布式存储系统的数据可用性。