权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

CNS: Core: Medium: Understanding and addressing device-reliability heterogeneity in large-scale distributed storage

CNS：核心：中：理解和解决大规模分布式存储中的设备可靠性异构性

基本信息

批准号：
1956271
负责人：
Gregory Ganger
金额：
$ 97.08万
依托单位：
Carnegie-Mellon University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-10-01 至 2024-03-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1956271&HistoricalAwards=false
关键词：
CNS Core Medium Understanding addressing

项目摘要

This project develops new data protection approaches to reduce both data losses and cost-inefficiency in large storage systems. Modern data-centers store data in large collections (10s of thousands) of disk storage devices. To avoid data loss in the event of a disk failure, each chunk of data is stored redundantly, such that there are multiple copies or extra code-information that allows data chunk reconstruction. Previous approaches assume that all of the disks are the same in terms of their failure rates and aging properties. But, while that was true for smaller storage systems in the past, it is not true for the large storage systems underlying today's data-driven world. The result is that some data is not protected enough and other data is protected too much (wasting money) - worse, no one really knows which is which. This project develops new tools for dynamically learning the failure characteristics of different disks, automatically determining the right levels of redundancy for those characteristics, and efficiently adapting redundancy levels as circumstances change in the system.By addressing a major flaw in how storage reliability is designed and managed, this project will make data protection stronger and less expensive. Both are crucial. Data protection is crucial, because data is core to the success of data-driven revolutions happening in medicine, science, politics, business, etc. Losing data hampers their success. Data storage cost is crucial, because of the massive dollar, power, and environmental effects of the disks and data-centers used to house and operate them. For example, early analyses suggest that this project's outcomes could potentially reduce the number of disks needed by 20%, which would represent millions fewer devices in U.S. data centers. This project will include engaging with companies and national labs that operate large storage systems in order to help move the research results into practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目开发了新的数据保护方法，以减少大型存储系统中的数据丢失和成本效率低下。现代数据中心将数据存储在磁盘存储设备的大集合（成千上万）中。为了避免在磁盘故障的情况下数据丢失，每个数据块都被冗余地存储，使得存在允许数据块重构的多个副本或额外的代码信息。以前的方法假设所有磁盘在故障率和老化特性方面都是相同的。但是，虽然这在过去对于较小的存储系统是正确的，但对于当今数据驱动世界的大型存储系统却不是这样。其结果是，一些数据没有得到足够的保护，而另一些数据受到了太多的保护（浪费金钱）-更糟糕的是，没有人真正知道哪个是哪个。该项目开发了新的工具，用于动态学习不同磁盘的故障特征，自动确定这些特征的正确冗余级别，并根据系统环境的变化有效地调整冗余级别。通过解决存储可靠性设计和管理中的一个主要缺陷，该项目将使数据保护更强大，成本更低。两者都至关重要。数据保护至关重要，因为数据是医学、科学、政治、商业等领域数据驱动革命成功的核心。数据存储成本是至关重要的，因为用于容纳和操作它们的磁盘和数据中心的巨额美元，电力和环境影响。例如，早期的分析表明，该项目的成果可能会将所需的磁盘数量减少20%，这意味着美国数据中心的设备数量将减少数百万台。该项目将包括与运营大型存储系统的公司和国家实验室合作，以帮助将研究成果付诸实践。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（10）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Bandwidth Cost of Code Conversions in the Split Regime

DOI：
10.48550/arxiv.2205.06793
发表时间：
2022-05
期刊：
2022 IEEE International Symposium on Information Theory (ISIT)
影响因子：
0
作者：
Francisco Maturana;K. V. Rashmi
通讯作者：
Francisco Maturana;K. V. Rashmi

Convertible Codes: Enabling Efficient Conversion of Coded Data in Distributed Storage

可转换代码：实现分布式存储中编码数据的高效转换

DOI：
发表时间：
2022
期刊：
IEEE transactions on information theory
影响因子：
2.5
作者：
Maturana, Francisco
通讯作者：
Maturana, Francisco

PACEMAKER: Avoiding HeART attacks in storage clusters with disk-adaptive redundancy

DOI：
发表时间：
2021-03
期刊：
影响因子：
0
作者：
Saurabh Kadekodi;Francisco Maturana;Suhas Jayaram Subramanya;Juncheng Yang;K. V. Rashmi;G. Ganger
通讯作者：
Saurabh Kadekodi;Francisco Maturana;Suhas Jayaram Subramanya;Juncheng Yang;K. V. Rashmi;G. Ganger

Bandwidth Cost of Code Conversions in Distributed Storage: Fundamental Limits and Optimal Constructions