CRII: CSR: Towards Pinpointing the Root Causes of Failures in Flash-based Storage Systems

存储金融交易、科学计算结果、家庭照片等的存储系统正在发生巨大变化。在闪存技术进步的推动下，几乎每一层存储系统都在积极优化，以充分利用闪存。这种趋势增加了系统的复杂性，并增加了出现模糊故障的机会。即使对于数据中心的专业人员来说，基于闪存的存储系统的故障诊断也是一项挑战，并且可能导致严重的停机甚至数据丢失。此外，在可预见的未来，随着闪存和其他非易失性存储技术的快速发展，故障诊断的难度将会增加。因此，迫切需要一个诊断整个系统并确定故障根源的框架。这个研究项目的目标是在基于闪存的存储系统中实现精确的端到端故障诊断。一个关键的观察是，越来越多的内核组件正在针对flash进行积极的优化，因此需要一个诊断框架来最大限度地减少对内核的依赖。另一方面，尽管许多组件都在变化，但一些基本的抽象（如文件和逻辑块）以及接口（如系统调用和命令集）相对稳定。诊断框架可以关注不变量来分析快速演化系统的行为。基于这些观察，本研究项目研究了一种新的故障诊断框架，该框架基于接口分离故障域，并自动关联各层的基本操作。更具体地说，该框架由两个协同组件组成：(1)驱动器上的设备代理，用于记录和重现主机-设备交互，而不依赖于主机；(2)端到端分析器，用于在从应用程序到设备的整个数据路径上提供深入的诊断支持。该研究项目将提高现代社会各种重要数据存储系统的可靠性。数据存储的改进将反过来从许多其他方面造福社会，包括减少维护成本和人力，避免由于服务停机或数据损坏而造成的经济损失。该研究项目还将有助于操作系统和其他相关课程的课程设置，并将吸引本科生和研究生参与系统研究。此外，该项目将被整合到少数民族参与联盟计划和年轻女性计算机计划中，以吸引传统上代表性不足的群体的学生，为我们国家、地区和地方在增加计算研究多样性方面的利益服务。

Storage systems that hold financial transactions, scientific computation results, a family's photos, and more, are undergoing dramatic changes. Triggered by the advances in flash memory technology, almost every layer of storage systems is being optimized aggressively to make full use of flash. This trend increases the complexity of the systems and increases the chance of obscure failures. Failures in flash-based storage systems are challenging to diagnose even for data center professionals, and can lead to severe downtime or even data loss. Moreover, the difficulty of failure diagnosis will increase in the foreseeable future with the rapid growth of flash and other non-volatile memory technologies. Thus, a framework for diagnosing the whole system and pinpointing the root causes of failures is urgently needed.The goal of this research project is to enable precise, end-to-end diagnosis of failures in flash-based storage systems. One key observation is that more and more kernel components are being optimized aggressively for flash, so a diagnosis framework is needed that minimizes the dependence on the kernel. On the other hand, although many components are changing, some fundamental abstractions such as files and logical blocks, and interfaces such as system calls and command sets, are relatively stable. A diagnosis framework can focus on the invariants to analyze the behavior of the rapidly evolving systems. Based on these observations, this research project investigates a novel failure diagnosis framework that separates failure domains based on the interfaces and automatically correlates the fundamental operations across layers. More specifically, the framework consists of two synergistic components: (1) an on-drive device agent to record and reproduce the host-device interactions without any dependence on the host, and (2) an end-to-end analyzer to provide in-depth diagnosis support along the whole data path from the application to the device.This research project will advance the dependability of storage systems for various important data in modern society. The improvements to data storage will in turn benefit the society from many other aspects, including examples such as reducing maintenance costs and human efforts and avoiding financial loss due to service downtime or data corruption. This research project will also contribute to the curriculum of operating systems and other related courses, and will engage undergraduate and graduate students in systems research. In addition, the project will be integrated into the Alliance for Minority Participation Program and the Young Women in Computing Program to engage students from traditionally underrepresented groups, serving our national, regional, and local interest in increasing diversity in computing research.