CRII: CSR: Towards Pinpointing the Root Causes of Failures in Flash-based Storage Systems

存储金融交易、科学计算结果、家庭照片等的存储系统正在经历巨大的变化。由于闪存技术的进步，存储系统的几乎每一层都在积极优化，以充分利用闪存。这种趋势增加了系统的复杂性，并增加了隐蔽故障的可能性。即使对于数据中心专业人员来说，基于闪存的存储系统中的故障也很难诊断，并且可能导致严重的停机甚至数据丢失。此外，在可预见的未来，随着闪存和其他非易失性存储器技术的快速发展，故障诊断的难度将增加。因此，迫切需要一个诊断整个系统并查明故障根源的框架。本研究项目的目标是实现基于闪存的存储系统中精确的端到端故障诊断。一个关键的观察结果是，越来越多的内核组件正在针对闪存进行积极的优化，因此需要一个诊断框架来最大限度地减少对内核的依赖。另一方面，尽管许多组件都在变化，但一些基本的抽象（如文件和逻辑块）以及接口（如系统调用和命令集）相对稳定。一个诊断框架可以专注于不变量来分析快速演化系统的行为。基于这些观察，本研究项目研究了一种新的故障诊断框架，该框架基于接口分离故障域，并自动关联跨层的基本操作。更具体地说，该框架由两个协同组件组成：（1）一个驱动器上的设备代理，以记录和再现主机-设备交互，而不依赖于主机，和（2）一个端到端的分析器，以提供深入的诊断支持沿整个数据路径从应用程序到设备沿着。对数据存储的改进反过来又会从许多其他方面使社会受益，包括减少维护成本和人力，以及避免由于服务停机或数据损坏而造成的经济损失。这个研究项目也将有助于操作系统和其他相关课程的课程，并将从事系统研究的本科生和研究生。此外，该项目将被纳入少数民族参与计划联盟和计算计划中的年轻女性，以吸引传统上代表性不足的群体的学生，为我们的国家，地区和地方利益增加计算研究的多样性。

Storage systems that hold financial transactions, scientific computation results, a family's photos, and more, are undergoing dramatic changes. Triggered by the advances in flash memory technology, almost every layer of storage systems is being optimized aggressively to make full use of flash. This trend increases the complexity of the systems and increases the chance of obscure failures. Failures in flash-based storage systems are challenging to diagnose even for data center professionals, and can lead to severe downtime or even data loss. Moreover, the difficulty of failure diagnosis will increase in the foreseeable future with the rapid growth of flash and other non-volatile memory technologies. Thus, a framework for diagnosing the whole system and pinpointing the root causes of failures is urgently needed.The goal of this research project is to enable precise, end-to-end diagnosis of failures in flash-based storage systems. One key observation is that more and more kernel components are being optimized aggressively for flash, so a diagnosis framework is needed that minimizes the dependence on the kernel. On the other hand, although many components are changing, some fundamental abstractions such as files and logical blocks, and interfaces such as system calls and command sets, are relatively stable. A diagnosis framework can focus on the invariants to analyze the behavior of the rapidly evolving systems. Based on these observations, this research project investigates a novel failure diagnosis framework that separates failure domains based on the interfaces and automatically correlates the fundamental operations across layers. More specifically, the framework consists of two synergistic components: (1) an on-drive device agent to record and reproduce the host-device interactions without any dependence on the host, and (2) an end-to-end analyzer to provide in-depth diagnosis support along the whole data path from the application to the device.This research project will advance the dependability of storage systems for various important data in modern society. The improvements to data storage will in turn benefit the society from many other aspects, including examples such as reducing maintenance costs and human efforts and avoiding financial loss due to service downtime or data corruption. This research project will also contribute to the curriculum of operating systems and other related courses, and will engage undergraduate and graduate students in systems research. In addition, the project will be integrated into the Alliance for Minority Participation Program and the Young Women in Computing Program to engage students from traditionally underrepresented groups, serving our national, regional, and local interest in increasing diversity in computing research.