XPS: FULL: Collaborative Research: Maximizing the Performance Potential and Reliability of Flash-based Solid State Devices for Future Storage Systems

基于NAND闪存的固态数据存储正在从根本上改变几乎整个信息技术基础设施的存储器和存储层次结构。尽管如此，在业界能够充分发掘闪存的潜力之前，仍有一些基本的和具有挑战性的问题需要解决。首先，随着闪存技术规模的缩小，其可靠性下降接近令人担忧的水平，导致许多应用中的存储系统架构师和用户的严重担忧和怀疑。其次，固态存储的系统和应用开发一直是独立进行的，导致这些层之间的隔离，重复操作和管理效率低下。由于现有的与存储设备的简单接口中的技术缩放和信息丢失，闪存在实践中尚未被有效和可靠地利用，并且随着技术缩放，情况将变得更糟。该项目的PI将采用整体系统设计方法，以解决阻碍闪存广泛采用的挑战。通过创新精心编排的跨层信息共享和利用，这种设计方法能够在整个软件/硬件堆栈中无缝利用系统级工作负载和物理级设备特性，而不会使整体系统设计复杂化。将开发一个集成的软件和硬件原型基础设施，以展示使用Hadoop、虚拟机和数据库等主要和广泛使用的软件系统的潜力。本项目将通过将基础研究成果转化为存储系统，培养本科生和研究生的研究活动，并及时将新的研究成果整合到课堂上，从而实现更广泛的影响。具体而言，本项目将开展几项密切相关的任务：（1）将开发能够学习和预测各个闪存设备的变化特性及其相关性的技术。这将提供运行时信息，使其能够优化闪存的使用，以缓解可靠性危机，并适应不同的系统级工作负载特性。(2)它将开发能够在存储层次结构中交换关键信息的技术，以促进跨层信息共享。(3)它将进一步开发一套跨设计层次的技术，可以有效地利用这些运行时收集和预测，以提高整体系统的可靠性和性能。

Solid-state data storage built upon NAND flash memory is fundamentally changing the memory and storage hierarchy for virtually the entire information technology infrastructure. Nevertheless, there have been several fundamental and challenging issues to be addressed before the industry can explore the flash memory to its full potential. First, as flash memory technology scales down, its reliability degradation approaches to an alarming level, leading to serious concerns and skepticism of storage system architects and users in many applications. Second, system and application development of solid-state storage has been independently conducted, resulting in isolation, duplicated operations, and an inefficient management among these layers. Due to the technology scaling and information loss in existing simple interface with storage devices, flash memory has not been efficiently and reliably utilized in practice, and the situation will become worse with the technology scaling. The PIs of this project will apply a holistic system design methodology to cohesively address the challenges preventing wider adoption of flash memory. By innovating well-orchestrated cross-layer information sharing and utilization, this design methodology enables seamless utilization of system-level workload and physical-level device characteristics across the entire software/hardware stack without complicating overall system design. An integrated software and hardware prototyping infrastructure will be developed to demonstrate the potential using major and widely used software systems, such as Hadoop, virtual machines, and database. This project will achieve a high broader impact by transforming basic research results into storage systems, and by training both undergraduate and graduate students with research activities, and by timely integrating new research results to classrooms.Specifically, this project will carry out several closely related tasks: (1) It will develop techniques that can learn and predict the varying characteristics and their correlations of individual flash memory devices. This will provide run-time information that makes it possible to optimize the use of flash memory for alleviating the reliability crisis and adapting to varying system-level workload characteristics. (2) It will develop techniques that enable critical information exchange across the storage hierarchy in order to facilitate cross-layer information sharing. (3) It will further develop a set of techniques across the design hierarchy that can effectively utilize these runtime collections and predictions to improve the overall system reliability and performance.