CAREER: Improving Storage System Performance, Dependability and Manageability Using System Mining Techniques

技术趋势表明，今天的计算正变得越来越以数据为中心。设备和服务的广泛使用产生了前所未有的信息存储和检索需求。目前存储需求的年增长率为60%。到2008年，数据中心平均管理的数据量将是现在的10倍。根据加州大学伯克利分校最近进行的一项研究，每年的存储需求大约是1.5艾字节，地球上每个人大约250兆字节。为了满足日益增长的数据服务需求，现代存储系统需要解决三个挑战：(1)性能，提供令人满意的性能以跟上快速增长的处理器速度；(2)可靠性，提供可靠性和可用性，以最大限度地减少数据访问损失，目前公司的数据访问损失超过每小时25万美元，占公司IT总预算的三分之一到一半；(3)可管理性，简化存储管理员的工作，降低存储维护成本，这几乎是目前存储设备采购价格的九倍。该提案解决了这三个挑战。它研究了一种称为系统挖掘的新技术，该技术将数据挖掘技术应用于存储系统，以提高其性能，可靠性和可管理性。具体而言，本文提出的系统基于以下创新：1)性能：使用频繁的序列挖掘、聚类、分类等数据挖掘算法来表征存储访问模式并推断数据语义，以指导存储缓存管理、预取、磁盘调度和数据布局，从而最大化存储性能；2)可靠性：对统一、关联的活动日志应用离群分析、签名分析等数据挖掘技术，发现并纠正存储管理员的错误和其他人为错误；可管理性：构建上下文感知、“自我成熟”的自主存储系统，该系统可以向存储管理员学习并自动生成管理脚本，从而逐步减少管理员的参与。

Technology trends indicate that today's computing is becoming more and more data-centric. The widespread use of devices and services has created unprecedented demand to store and retrieve information. The current annual growth of storage demand is 60%. By 2008, the average data center will manage 10 times as much data as it does today. According to a recent study conducted by UC Berkeley, the annual storage demand is roughly 1.5 exabytes of storage, around 250 megabytes per person for everyone on earth.To satisfy the increasing data service demand, modern storage systems need to address three challenges: (1) performance, delivering satisfactory performance to keep up with the rapid growing processor speed; (2) dependability, providing reliability and availability to minimize data access loss, which currently costs companies more that $250,000/hour and one-third to one-half of a company's total IT budget; (3) manageability, simplifying storage administrator's jobs to reduce the storage maintenance cost, which is currently almost nine times the storage equipment purchase price.This proposal addresses these three challenges. It investigates a novel technology called system mining that applies data mining techniques to storage systems to improve their performance, dependability and manageability. More specifically, the proposed system hinges on the following innovations:1) Performance: using frequent sequence mining, clustering, classification and other data mining algorithms to characterize storage access patterns and infer data semantics for guiding storage cache management, prefetching, disk scheduling, and data layout to maximize storage performance;2) Dependability: applying outlier analysis, signature analysis and other data mining techniques to unified, correlated activity logs to detect and correct storage administrators' mistakes and other human errors;Manageability: building a context-aware, "self-maturing" autonomic storage system that can learn from storage administrators and automatically generate administrative scripts to gradually minimize administrators' involvement.