CRII: CSR: Towards Pinpointing the Root Causes of Failures in Flash-based Storage Systems

存储金融交易、科学计算结果、家庭照片等的存储系统正在经历巨大的变化。在闪存技术进步的推动下，几乎每一层存储系统都在积极优化，以充分利用闪存。这种趋势增加了系统的复杂性，增加了隐蔽故障的可能性。即使对于数据中心专业人员来说，基于闪存的存储系统中的故障也很难诊断，可能会导致严重的停机甚至数据丢失。此外，在可预见的未来，随着闪存和其他非易失性存储器技术的快速发展，故障诊断的难度将增加。因此，迫切需要一个框架来诊断整个系统并找出故障的根本原因。本研究项目的目标是能够对基于闪存的存储系统进行精确的端到端故障诊断。一个关键的观察是，越来越多的内核组件正在针对闪存进行积极的优化，因此需要一个诊断框架来最大限度地减少对内核的依赖。另一方面，尽管许多组件在变化，但一些基本的抽象，如文件和逻辑块，以及接口，如系统调用和命令集，都是相对稳定的。诊断框架可以将重点放在不变量上，以分析快速演化系统的行为。基于这些观察结果，本研究项目研究了一种新的故障诊断框架，该框架基于接口分离故障域，并自动关联各层的基本操作。该框架由两个协同组件组成：(1)驱动设备代理，用于记录和再现主机与设备的交互，而不依赖于主机；(2)端到端分析器，用于在从应用程序到设备的整个数据路径上提供深入的诊断支持。数据存储的改进将反过来从许多其他方面造福社会，包括减少维护成本和人力努力，以及避免因服务停机或数据损坏而造成的经济损失。该研究项目还将有助于操作系统课程和其他相关课程，并将使本科生和研究生参与系统研究。此外，该项目将被整合到少数群体参与计划和年轻女性参与计算计划，以吸引来自传统上代表性不足的群体的学生，服务于我们在增加计算研究多样性方面的国家、地区和地方利益。

Storage systems that hold financial transactions, scientific computation results, a family's photos, and more, are undergoing dramatic changes. Triggered by the advances in flash memory technology, almost every layer of storage systems is being optimized aggressively to make full use of flash. This trend increases the complexity of the systems and increases the chance of obscure failures. Failures in flash-based storage systems are challenging to diagnose even for data center professionals, and can lead to severe downtime or even data loss. Moreover, the difficulty of failure diagnosis will increase in the foreseeable future with the rapid growth of flash and other non-volatile memory technologies. Thus, a framework for diagnosing the whole system and pinpointing the root causes of failures is urgently needed.The goal of this research project is to enable precise, end-to-end diagnosis of failures in flash-based storage systems. One key observation is that more and more kernel components are being optimized aggressively for flash, so a diagnosis framework is needed that minimizes the dependence on the kernel. On the other hand, although many components are changing, some fundamental abstractions such as files and logical blocks, and interfaces such as system calls and command sets, are relatively stable. A diagnosis framework can focus on the invariants to analyze the behavior of the rapidly evolving systems. Based on these observations, this research project investigates a novel failure diagnosis framework that separates failure domains based on the interfaces and automatically correlates the fundamental operations across layers. More specifically, the framework consists of two synergistic components: (1) an on-drive device agent to record and reproduce the host-device interactions without any dependence on the host, and (2) an end-to-end analyzer to provide in-depth diagnosis support along the whole data path from the application to the device.This research project will advance the dependability of storage systems for various important data in modern society. The improvements to data storage will in turn benefit the society from many other aspects, including examples such as reducing maintenance costs and human efforts and avoiding financial loss due to service downtime or data corruption. This research project will also contribute to the curriculum of operating systems and other related courses, and will engage undergraduate and graduate students in systems research. In addition, the project will be integrated into the Alliance for Minority Participation Program and the Young Women in Computing Program to engage students from traditionally underrepresented groups, serving our national, regional, and local interest in increasing diversity in computing research.