CSR: Medium: Collaborative Research: Wizard: Exploiting Disk Performance Signatures for Cost-Effective Management of Large-Scale Storage Systems

低成本、大容量磁盘驱动器、闪存固态驱动器和非易失性存储器的巨大进步，一直是支持现代社会深深依赖的大数据应用和各种计算存储服务的关键因素。然而，据报道，由于故障，存储驱动器是最常更换的硬件组件。这会导致服务中断甚至数据丢失，每年给企业造成数万亿美元的损失。现有的磁盘故障管理方法大多是反应性的，并且会产生较高的管理费用；它们不能为管理大规模生产存储系统提供经济高效的解决方案。本项目的目标是深入了解现实世界存储系统的可靠性，并开发具有成本效益的数据和存储资源管理系统，以提高可靠性。研究人员构建可靠、大规模存储系统的方法经过精心设计，以系统的方式支持存储运行状况监控、建模、预测和主动恢复。特别是，他们首先对存储故障进行分类和建模，以得出磁盘性能特征，并探索磁盘性能特征，以预测磁盘故障的发生。然后，他们表征磁盘性能下降中的I/O工作负载依赖关系，并集成异类磁盘设备的性能签名，以有效地重新配置和管理存储资源。此外，该项目还将为存储用户和开发人员提供简单易用的API，以使用开发的工具和技术进行主动数据救援和预防性磁盘可靠性增强。最后，该项目为培训研究生，特别是少数族裔和女性学生，以及编写关于可靠存储系统的新课程材料提供了极好的机会。

The tremendous advances in low-cost, high-capacity magnetic hard disk drives, flash-based solid state drives and non-volatile memory have been among the key factors supporting big data applications and various computing-storage services that the modern society deeply relies on. However, storage drives are reported to be the most commonly replaced hardware components because of failures. This causes service downtime and even data loss, costing enterprises multi-trillion dollars per year. Existing disk failure management approaches are mostly reactive and incur high overheads; they do not provide a cost-effective solution to managing large-scale production storage systems. The goal of this project is to achieve a deep understanding of the reliability of the real-world storage systems, and to develop a cost-effective data and storage resource management system for reliability enhancement. The investigators' approach to building reliable, large-scale storage systems is carefully designed to support storage health monitoring, modeling, prediction and proactive recovery in a systematic fashion. In particular, they first categorize and model storage failures to derive disk performance signatures and explore disk performance signatures to forecast occurrences of disk failures. They then characterize I/O workload dependency in disk performance degradation and integrate the performance signatures of heterogeneous disk devices to effectively reconfigure and manage storage resources. Furthermore, the project will provide easy-to-use APIs for storage users and developers to employ the developed tools and techniques for proactive data rescue and preventive disk reliability enhancement. Finally, the project provides excellent opportunities for training graduate students, especially minority and female students, and for developing new curriculum materials on reliable storage systems.