SHF: Medium: Breaking the Physical Divide between Computation and NAND-Flash Storage

如今，几乎所有应用程序细分市场都经历着数据爆炸，这意味着它们需要以一种同时具有性能感知和能源感知的方式存储、访问、操作和转换存储在不同介质中的海量数据。这些渴望数据的细分市场包括(I)移动和家用电子产品领域的消费应用，(Ii)提供丰富内容和用户体验的桌面应用，(Iii)产生PB级数据以分析在时间和空间尺度上前所未闻的实验和真实世界现象的科学应用，(Iv)不知疲倦地存储用于审计、分析和优化的各种数据/知识的企业应用，(V)使用存储来保存工作负荷的大虚拟机映像以跨不同服务器进行整合的数据中心和云平台，(Vi)需要存储的互联网服务和社交网络平台，跟踪和管理用户模式，以及(Vii)持续感知和存储物理世界数据以进行实时分析和控制的网络物理应用程序。目前的计算机基础设施装备不足，无法满足这种数据需求。造成这种情况的主要原因是计算和存储之间固有的物理鸿沟。虽然计算和存储技术在过去几十年中都有了巨大的改进，但它们之间的交互和接口并没有，从而限制了关键数据密集型应用程序的性能。如果不及时解决，这个问题有可能减缓科学发现和工程突破。该项目通过打破计算和NAND闪存之间的物理鸿沟来解决数据管理问题。这样做可以潜在地允许计算和存储之间的通信带宽与计算资源和存储资源的并行性驱动的扩展一起扩展。它还可以让双方更好地了解对方的意图和操作，从而为更科学地管理存储提供更多的可能性。随着应用程序开始施加更严格的计算和存储需求，这反过来将允许两者更好地共同设计、共同管理和共同发展，从而在未来实现更好的可扩展性。具体地说，该项目研究了三种弥合计算和NAND之间的物理鸿沟的主要策略-闪存存储。第一种策略实现了闪存与主机之间更好的协作；第二种策略将NAND-Flash存储提升为直接与处理器接口，类似于通过内存控制器接口到片上核心的主内存DIMM(双列直插式内存模块)；最后一种策略探索了不同的布局选项，以更紧密地集成NAND-Flash存储与计算资源。这项研究的更广泛影响包括学生培训、代表性不足群体的参与、招聘研讨会、将教育模块纳入现有和未来的课程以及公共领域模拟工具。此外，通过工程周末访问(VIEW)计划，该项目培养了人们对计算机科学和工程的兴趣。该项目提供动手设计活动，以激励与存储系统和数据管理有关的计算机科学和工程新领域的参与者。

Almost all application segments today experience data explosion, meaning that they need to store, access, manipulate and transform extremely large amounts of data stored in different mediums in a fashion that is simultaneously performance-aware and energy-aware. These data-hungry market segments include (i) consumer applications in the mobile and home electronics segment, (ii) desktop applications that are providing rich content and user experience, (iii) scientific applications that generate petabytes of data for analyzing experiments and real-world phenomena on temporal and spatial scales unheard of before, (iv) enterprise applications which tirelessly store all kinds of data/knowledge for auditability, analytics, and optimization, (v) datacenters and cloud platforms which use storage to hold large virtual machine images of the workloads for consolidation across different servers, (vi) Internet services and social networking platforms which need to store, track and manage user patterns, and (vii) cyber-physical applications which continuously sense and store physical world data for real-time analytics and control. Current computer infrastructures are poorly equipped to cope with this data demand. The primary reason for this is the inherent physical divide between computation and storage. While both computation and storage technologies have undergone tremendous improvements in the last decades, the interactions and interfaces between them have not, thereby limiting the performance of critical data-intensive applications. If not addressed in a timely fashion, this problem has the potential to slow down scientific discoveries and engineering breakthroughs. This project addresses the data management problem by breaking the physical divide between computation and NAND-flash storage. Doing so can potentially allow the communication bandwidth between computation and storage to scale together with the parallelism-driven scaling of both computation resources and storage resources. It can also allow each to become more aware of the intentions and operations of the other, opening a wide spectrum of possibilities in more efficiently managing storage. This will in turn allow better co-design, co-management, and co-evolution of the two for better scalability in the future, as applications start imposing even more stringent computing and storage demands. Specifically, this project investigates three main strategies for bridging the physical divide between compute and NAND-flash storage. The first strategy enables better cooperation between flash storage and host; the second strategy elevates NAND-flash storage to directly interface with the processors, similar to main memory DIMMs (dual inline memory modules) interfacing to the on-chip cores through memory controllers; and the last strategy explores different placement options for tighter integration of NAND-flash storage with computational resources. The broader impacts of this research include student training, participation of under-represented groups, recruiting workshops, incorporation of educational modules into existing and future courses, and public domain simulation tools. Further, through the Visit In Engineering Weekend (VIEW) program, the project fosters interest in computer science and engineering. The project provides hands-on-design activities to motivate the VIEW participants in new areas of computer science and engineering related to storage system and data management.