CIF: Small: Distributed Storage Systems from Combinatorial Designs

CIF：小型：组合设计的分布式存储系统

• 批准号: 1320416
• 负责人: Aditya Ramamoorthy
• 金额: $ 40万
• 依托单位: Iowa State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-08-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1320416&HistoricalAwards=false
• 关键词: CIF; Small; Distributed; Storage; Systems

Large scale content sharing, social networking sites and cloud storage providers require massive amounts of data (on the order of hundreds of petabytes) to be stored in a distributed manner. In these systems, node failures and loss of data are the norm rather than the exception. Thus, it is of critical importance to investigate storage techniques and algorithms that (a) allow for data recovery in the presence of node failures and (b) allow for efficient regeneration of failed nodes. It is, of course, desirable to perform the regeneration in a distributed manner and optimize performance metrics associated with it. For instance, it is desirable that the regeneration process be fast and that only a few nodes be contacted for the repair. The investigator will research storage methods that allow for very simple and efficient repair while simultaneously allowing for data recovery in the presence of a large number of failures. The research involves the investigation of distributed storage systems that are resilient to multiple failures and can be repaired in an exact and uncoded manner; i.e., the new node may produce an exact copy of the failed node by simply downloading packets from the surviving nodes, with no computational overhead. Replication codes are increasingly deployed in large-scale data centers in use by major internet companies. Advances in reconstruction codes are expected to translate into millions of dollars in savings in maintenance costs in such data centers. To ensure accessibility of the results of this research, implementation will be carried out using the open source Hadoop distributed file system, the preferred platform for cutting-edge research and development in distributed cloud storage.

大规模内容共享、社交网站和云存储提供商需要以分布式方式存储大量数据（大约数百PB）。在这些系统中，节点故障和数据丢失是常态而不是例外。因此，它是至关重要的调查存储技术和算法，（a）允许在节点故障的存在下的数据恢复和（B）允许故障节点的有效再生。当然，希望以分布式方式执行再生并优化与之相关联的性能度量。例如，希望再生过程快速并且仅联系几个节点进行修复。研究人员将研究存储方法，允许非常简单和有效的修复，同时允许在大量故障的情况下恢复数据。该研究涉及对分布式存储系统的调查，这些系统对多个故障具有弹性，并且可以以精确和未编码的方式进行修复;即，新节点可以通过简单地从幸存节点下载分组而产生故障节点的精确副本，而没有计算开销。复制代码越来越多地部署在大型互联网公司使用的大型数据中心中。重建代码的进步预计将转化为数百万美元的维护成本节省在这样的数据中心。为了确保这项研究的成果的可访问性，将使用开源Hadoop分布式文件系统进行实施，这是分布式云存储尖端研究和开发的首选平台。