SPX: Collaborative Research: Scaling the Software-Defined Data Center with Network-Storage Stack Co-Design

SPX：协作研究：通过网络存储堆栈协同设计扩展软件定义的数据中心

• 批准号: 1919044
• 负责人: Jian Huang
• 金额: $ 43.89万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1919044&HistoricalAwards=false
• 关键词: SPX; Collaborative; Research; Scaling; Software

Computer data centers today are experiencing a disruptive trend by virtue of becoming software-defined and programmable. Software-defined networking (SDN) and software-defined storage (SDS) have been widely deployed in today's data centers to provide multi-tenant network and storage services. However, due to the lack of coordination between the network and storage software, there are still performance, scalability, and reliability challenges in data centers. As the data center is evolving with software-defined functions (hitherto done through hardware devices), it is the right time to fundamentally address these classical challenges. This project leverages a cross-layer research approach to conduct the fundamental systems and architecture study on the software-defined network and storage. The proposed research activities will benefit the infrastructure and platform development for large-scale data centers. The project will involve active industrial collaborations to carve a path for technology transition to practice. Furthermore, the project will integrate the research developments into new courses that center around network and storage techniques. This project will also recruit and train underrepresented and high school students through various research programs and summer camps.This project investigates a new software-defined architecture that co-designs the system stacks of both SDN and SDS while refining the functionalities of each component. If successful, the project will fundamentally advance design principles and techniques of software-defined data centers. First, the project proposes to refine the functionalities of the control and data plane in SDN and SDS. Such a new architecture enables state sharing between SDN and SDF, which facilitates the global resource management and I/O scheduling to address the performance challenge. Second, to overcome scalability challenges, this project rethinks the design and implementation of concurrency control in the storage stack and proposes an in-network lock manager to scale the concurrent storage accesses. The project further exploits the hardware parallelism available in multi-core CPU chips to speed up the network packet processing. Third, the project analyzes the limitation of existing data replication protocols and proposes an in-network conflict detection scheme to achieve both linear scalability and strong data consistency for replicated storage. Finally, the project applies the proposed techniques to representative multi-tenant storage services such as distributed databases and key-value stores, and new distributed applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

由于软件定义和可编程，今天的计算机数据中心正在经历一种颠覆性的趋势。软件定义网络（SDN）和软件定义存储（SDS）已广泛部署在当今的数据中心中，以提供多租户网络和存储服务。然而，由于网络和存储软件之间缺乏协调，数据中心在性能、可扩展性和可靠性方面仍然存在挑战。随着数据中心随着软件定义功能的发展（迄今为止通过硬件设备完成），现在是从根本上解决这些经典挑战的合适时机。本项目采用跨层研究方法，对软件定义网络和存储进行基础系统和架构研究。所提出的研究活动将有利于大型数据中心的基础设施和平台开发。该项目将涉及积极的工业合作，为技术过渡到实践开辟一条道路。此外，该项目将把研究成果整合到以网络和存储技术为中心的新课程中。该项目还将通过各种研究项目和夏令营招募和培训代表性不足的高中生。该项目研究了一种新的软件定义架构，该架构在细化每个组件的功能的同时，共同设计SDN和SDS的系统堆栈。如果成功，该项目将从根本上推进软件定义数据中心的设计原则和技术。首先，本项目提出在SDN和SDS中细化控制平面和数据平面的功能。这样的新架构使SDN和SDF之间的状态共享成为可能，这有利于全局资源管理和I/O调度，以应对性能挑战。其次，为了克服可扩展性的挑战，本项目重新思考了存储栈并发控制的设计和实现，并提出了一种网内锁管理器来扩展并发存储访问。该项目进一步利用多核CPU芯片中的硬件并行性来加速网络数据包处理。第三，分析了现有数据复制协议的局限性，提出了一种网络内冲突检测方案，以实现复制存储的线性可扩展性和强数据一致性。最后，该项目将提出的技术应用于代表性的多租户存储服务，如分布式数据库和键值存储，以及新的分布式应用程序。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

UniHeap: managing persistent objects across managed runtimes for non-volatile memory

UniHeap：跨托管运行时管理非易失性内存的持久对象

• DOI: 10.1145/3456727.3463775
• 发表时间: 2021
• 期刊: SYSTOR '21: Proceedings of the 14th ACM International Conference on Systems and Storage
• 作者: Li, Daixuan;Reidys, Benjamin;Sun, Jinghan;Shull, Thomas;Torrellas, Josep;Huang, Jian
• 通讯作者: Huang, Jian

BlockFlex: Enabling Storage Harvesting with Software-Defined Flash in Modern Cloud Platforms

• 发表时间: 2022
• 作者: Benjamin Reidys;Jinghan Sun;Anirudh Badam;S. Noghabi;Jian Huang

Distributed Data Persistency

• DOI: 10.1145/3466752.3480060
• 发表时间: 2021-10
• 期刊: IEEE Micro
• 影响因子: 3.6
• 作者: Apostolos Kokolis;Antonis Psistakis;Benjamin Reidys;Jian Huang;J. Torrellas

Pinpointing crash-consistency bugs in the HPC I/O stack: a cross-layer approach

查明 HPC I/O 堆栈中的崩溃一致性错误：跨层方法

• 发表时间: 2021
• 期刊: Storage and Analysis (SC21
• 作者: Sun, J.;Huang, J.;Snir M.
• 通讯作者: Snir M.

Building Next-Generation Software-Defined Data Centers with Network-Storage Co-design

通过网络存储协同设计构建下一代软件定义的数据中心

• 发表时间: 2023
• 期刊: The 1st Workshop on Hot Topics in System Infrastructure (HotInfra'23
• 作者: Reidys Benjamin;Huang, Jian.
• 通讯作者: Huang, Jian.