SPX: Collaborative Research: Scaling the Software-Defined Data Center with Network-Storage Stack Co-Design

当今的计算机数据中心正在经历一种颠覆性的趋势，因为它变得软件定义和可编程。软件定义网络(SDN)和软件定义存储(SDS)已广泛部署在当今的数据中心，以提供多租户网络和存储服务。然而，由于网络和存储软件之间缺乏协调，数据中心仍然存在性能、可扩展性和可靠性方面的挑战。随着数据中心使用软件定义的功能(迄今通过硬件设备完成)的发展，现在是从根本上解决这些经典挑战的合适时机。该项目利用跨层研究方法对软件定义的网络和存储进行基础系统和体系结构研究。建议的研究活动将有利于大型数据中心的基础设施和平台开发。该项目将涉及积极的行业合作，为技术转化为实践开辟一条道路。此外，该项目将把研究进展整合到围绕网络和存储技术的新课程中。该项目还将通过各种研究计划和夏令营招募和培训代表不足的高中生。该项目研究一种新的软件定义的体系结构，该体系结构共同设计SDN和SDS的系统堆栈，同时完善每个组件的功能。如果成功，该项目将从根本上推进软件定义的数据中心的设计原则和技术。首先，该项目建议完善SDN和SDS中控制和数据平面的功能。这种新的体系结构实现了SDN和SDF之间的状态共享，从而促进了全局资源管理和I/O调度，以应对性能挑战。其次，为了克服可伸缩性挑战，该项目重新考虑了存储堆栈中并发控制的设计和实现，并提出了一个网络内锁管理器来扩展并发存储访问。该项目进一步利用了多核CPU芯片的硬件并行性来加速网络报文的处理。第三，分析了现有数据复制协议的局限性，提出了一种网络内冲突检测方案，以实现复制存储的线性可伸缩性和强数据一致性。最后，该项目将建议的技术应用于具有代表性的多租户存储服务，如分布式数据库和键值存储，以及新的分布式应用程序。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Computer data centers today are experiencing a disruptive trend by virtue of becoming software-defined and programmable. Software-defined networking (SDN) and software-defined storage (SDS) have been widely deployed in today's data centers to provide multi-tenant network and storage services. However, due to the lack of coordination between the network and storage software, there are still performance, scalability, and reliability challenges in data centers. As the data center is evolving with software-defined functions (hitherto done through hardware devices), it is the right time to fundamentally address these classical challenges. This project leverages a cross-layer research approach to conduct the fundamental systems and architecture study on the software-defined network and storage. The proposed research activities will benefit the infrastructure and platform development for large-scale data centers. The project will involve active industrial collaborations to carve a path for technology transition to practice. Furthermore, the project will integrate the research developments into new courses that center around network and storage techniques. This project will also recruit and train underrepresented and high school students through various research programs and summer camps.This project investigates a new software-defined architecture that co-designs the system stacks of both SDN and SDS while refining the functionalities of each component. If successful, the project will fundamentally advance design principles and techniques of software-defined data centers. First, the project proposes to refine the functionalities of the control and data plane in SDN and SDS. Such a new architecture enables state sharing between SDN and SDF, which facilitates the global resource management and I/O scheduling to address the performance challenge. Second, to overcome scalability challenges, this project rethinks the design and implementation of concurrency control in the storage stack and proposes an in-network lock manager to scale the concurrent storage accesses. The project further exploits the hardware parallelism available in multi-core CPU chips to speed up the network packet processing. Third, the project analyzes the limitation of existing data replication protocols and proposes an in-network conflict detection scheme to achieve both linear scalability and strong data consistency for replicated storage. Finally, the project applies the proposed techniques to representative multi-tenant storage services such as distributed databases and key-value stores, and new distributed applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.