CNS: Core: Medium: Understanding and addressing device-reliability heterogeneity in large-scale distributed storage

该项目开发了新的数据保护方法，以减少大型存储系统中的数据丢失和成本效率低下。现代数据中心将数据存储在磁盘存储设备的大集合（成千上万）中。 为了避免在磁盘故障的情况下数据丢失，每个数据块都被冗余地存储，使得存在允许数据块重构的多个副本或额外的代码信息。 以前的方法假设所有磁盘在故障率和老化特性方面都是相同的。 但是，虽然这在过去对于较小的存储系统是正确的，但对于当今数据驱动世界的大型存储系统却不是这样。 其结果是，一些数据没有得到足够的保护，而另一些数据受到了太多的保护（浪费金钱）-更糟糕的是，没有人真正知道哪个是哪个。该项目开发了新的工具，用于动态学习不同磁盘的故障特征，自动确定这些特征的正确冗余级别，并根据系统环境的变化有效地调整冗余级别。通过解决存储可靠性设计和管理中的一个主要缺陷，该项目将使数据保护更强大，成本更低。 两者都至关重要。 数据保护至关重要，因为数据是医学、科学、政治、商业等领域数据驱动革命成功的核心。 数据存储成本是至关重要的，因为用于容纳和操作它们的磁盘和数据中心的巨额美元，电力和环境影响。例如，早期的分析表明，该项目的成果可能会将所需的磁盘数量减少20%，这意味着美国数据中心的设备数量将减少数百万台。该项目将包括与运营大型存储系统的公司和国家实验室合作，以帮助将研究成果付诸实践。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project develops new data protection approaches to reduce both data losses and cost-inefficiency in large storage systems. Modern data-centers store data in large collections (10s of thousands) of disk storage devices. To avoid data loss in the event of a disk failure, each chunk of data is stored redundantly, such that there are multiple copies or extra code-information that allows data chunk reconstruction. Previous approaches assume that all of the disks are the same in terms of their failure rates and aging properties. But, while that was true for smaller storage systems in the past, it is not true for the large storage systems underlying today's data-driven world. The result is that some data is not protected enough and other data is protected too much (wasting money) - worse, no one really knows which is which. This project develops new tools for dynamically learning the failure characteristics of different disks, automatically determining the right levels of redundancy for those characteristics, and efficiently adapting redundancy levels as circumstances change in the system.By addressing a major flaw in how storage reliability is designed and managed, this project will make data protection stronger and less expensive. Both are crucial. Data protection is crucial, because data is core to the success of data-driven revolutions happening in medicine, science, politics, business, etc. Losing data hampers their success. Data storage cost is crucial, because of the massive dollar, power, and environmental effects of the disks and data-centers used to house and operate them. For example, early analyses suggest that this project's outcomes could potentially reduce the number of disks needed by 20%, which would represent millions fewer devices in U.S. data centers. This project will include engaging with companies and national labs that operate large storage systems in order to help move the research results into practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.