SHF: Medium: Breaking the Physical Divide between Computation and NAND-Flash Storage

如今，几乎所有的应用领域都经历着数据爆炸，这意味着它们需要以一种同时具有性能意识和能量意识的方式存储、访问、操作和转换存储在不同介质中的海量数据。这些数据需求巨大的细分市场包括(i)移动和家用电子领域的消费者应用程序，（ii）提供丰富内容和用户体验的桌面应用程序，（iii）生成pb级数据的科学应用程序，用于分析以前闻所未闻的时间和空间尺度上的实验和现实世界现象，（iv）不知厌倦地存储各种数据/知识以供审计、分析和优化的企业应用程序。(v)使用存储来保存工作负载的大型虚拟机映像以便跨不同服务器进行整合的数据中心和云平台；（vi）需要存储、跟踪和管理用户模式的互联网服务和社交网络平台；以及（vii）持续感知和存储物理世界数据以进行实时分析和控制的网络物理应用程序。目前的计算机基础设施不足以应付这种数据需求。造成这种情况的主要原因是计算和存储之间固有的物理划分。虽然计算和存储技术在过去几十年中都有了巨大的改进，但它们之间的交互和接口却没有，从而限制了关键数据密集型应用程序的性能。如果不及时解决，这个问题有可能减缓科学发现和工程突破。该项目通过打破计算和nand闪存存储之间的物理鸿沟来解决数据管理问题。这样做可以潜在地允许计算和存储之间的通信带宽与并行驱动的计算资源和存储资源的扩展一起扩展。它还可以让每个人都更加了解对方的意图和行动，以更多的方式打开更广泛的可能性。有效地管理存储。当应用程序开始施加更严格的计算和存储需求时，这将反过来允许两者更好地协同设计、协同管理和协同发展，从而在未来获得更好的可伸缩性。具体来说，这个项目研究了三种主要的策略来弥合计算和nand闪存存储之间的物理鸿沟。第一种策略使闪存和主机之间更好地协作；第二种策略将nand闪存提升到直接与处理器接口，类似于主存储器dimm（双内联存储器模块）通过存储器控制器与片上核心接口；最后一个策略探讨了nand闪存存储与计算资源更紧密集成的不同放置选项。这项研究的广泛影响包括学生培训、代表性不足的群体的参与、招聘讲习班、将教育模块纳入现有和未来的课程以及公共领域的模拟工具。此外，通过工程周末访问（VIEW）计划，该项目培养了对计算机科学和工程的兴趣。该项目提供了动手设计活动，以激励VIEW参与者在与存储系统和数据管理相关的计算机科学和工程的新领域。

Almost all application segments today experience data explosion, meaning that they need to store, access, manipulate and transform extremely large amounts of data stored in different mediums in a fashion that is simultaneously performance-aware and energy-aware. These data-hungry market segments include (i) consumer applications in the mobile and home electronics segment, (ii) desktop applications that are providing rich content and user experience, (iii) scientific applications that generate petabytes of data for analyzing experiments and real-world phenomena on temporal and spatial scales unheard of before, (iv) enterprise applications which tirelessly store all kinds of data/knowledge for auditability, analytics, and optimization, (v) datacenters and cloud platforms which use storage to hold large virtual machine images of the workloads for consolidation across different servers, (vi) Internet services and social networking platforms which need to store, track and manage user patterns, and (vii) cyber-physical applications which continuously sense and store physical world data for real-time analytics and control. Current computer infrastructures are poorly equipped to cope with this data demand. The primary reason for this is the inherent physical divide between computation and storage. While both computation and storage technologies have undergone tremendous improvements in the last decades, the interactions and interfaces between them have not, thereby limiting the performance of critical data-intensive applications. If not addressed in a timely fashion, this problem has the potential to slow down scientific discoveries and engineering breakthroughs. This project addresses the data management problem by breaking the physical divide between computation and NAND-flash storage. Doing so can potentially allow the communication bandwidth between computation and storage to scale together with the parallelism-driven scaling of both computation resources and storage resources. It can also allow each to become more aware of the intentions and operations of the other, opening a wide spectrum of possibilities in more efficiently managing storage. This will in turn allow better co-design, co-management, and co-evolution of the two for better scalability in the future, as applications start imposing even more stringent computing and storage demands. Specifically, this project investigates three main strategies for bridging the physical divide between compute and NAND-flash storage. The first strategy enables better cooperation between flash storage and host; the second strategy elevates NAND-flash storage to directly interface with the processors, similar to main memory DIMMs (dual inline memory modules) interfacing to the on-chip cores through memory controllers; and the last strategy explores different placement options for tighter integration of NAND-flash storage with computational resources. The broader impacts of this research include student training, participation of under-represented groups, recruiting workshops, incorporation of educational modules into existing and future courses, and public domain simulation tools. Further, through the Visit In Engineering Weekend (VIEW) program, the project fosters interest in computer science and engineering. The project provides hands-on-design activities to motivate the VIEW participants in new areas of computer science and engineering related to storage system and data management.