NeTS: CSR: Large: Collaborative Research: Co-Design of Network, Storage and Computation Fabrics for Disaggregated Datacenters

NeTS：CSR：大型：协作研究：分解数据中心的网络、存储和计算结构的协同设计

• 批准号: 1704742
• 负责人: Rachit Agarwal
• 金额: $ 209.99万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-15 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1704742&HistoricalAwards=false
• 关键词: NeTS; CSR; Large; Collaborative; Research

Traditional datacenters are built using servers, each of which tightly integrates a small amount of CPU (central processing unit), memory and storage onto a single motherboard. However, the end of Dennard's scaling and the slowdown of Moore's Law has led to surfacing of several fundamental limitations of such server-centric architectures (e.g., the memory-capacity wall making CPU-memory co-location unsustainable). Consequently, a new computing paradigm is emerging -- a disaggregated architecture, where each resource type is built as a standalone 'blade' and a network fabric interconnects the resource blades within and across racks. The computer architecture community has established a number of benefits of such disaggregated architectures, including the potential to have 10-100x larger resource capacity. While beneficial from the computer architecture perspective, disaggregated architectures alter several assumptions that once guided the design and optimization of existing networks, systems and applications (e.g., CPU-storage colocation, high CPU-memory bandwidth, storage hierarchy, data locality, failure models, etc.). Capitalizing on the benefits of disaggregated architectures will thus require re-architecting legacy systems and networks.This project aims to co-design the network, storage and compute fabrics for disaggregated datacenters.On the network front, the project will design ultra-low latency intra-rack and inter-rack fabrics including a new network software stack that incorporates efficient congestion control, failure tolerance and scheduling mechanisms. The co-design of network and storage fabrics will lead to new (distributed) memory and storage management stacks for disaggregated storage, and a resource manager that provides essential isolation, sharing and elasticity guarantees across multiple applications sharing disaggregated storage and network fabrics. Finally, the project will build new distributed programming frameworks and re-architect existing applications to efficiently and correctly operate on disaggregated architectures. This project will provide solutions to some of the most difficult and important technical questions surrounding this emerging computing paradigm and will have broad community impact primarily through educational and outreach activities, and technology transfer. Software artifacts resulting from this project will be publicly released to ensure repeatability and to foster follow up research. The project also has a substantial educational component including new courses and public release of teaching materials. Finally, the project will provide the necessary thrust to build an inter-disciplinary research community via mentoring of graduate students and postdoctoral scholars, yearly workshops and industry retreats to bridge the gap between industrial development and academic research.

传统的数据中心是使用服务器构建的，每个服务器都将少量的CPU(中央处理器)、内存和存储紧密集成到单个主板上。然而，Dennard‘s Scaling的终结和Moore’s Law的放缓导致这种以服务器为中心的体系结构的几个基本限制浮出水面(例如，内存容量墙使CPU-内存托管不可持续)。因此，一种新的计算范式正在出现--一种分散的体系结构，其中每种资源类型被构建为独立的“刀片”，并且网络交换矩阵将机架内和机架之间的资源刀片互连。计算机体系结构社区已经确定了这种分散的体系结构的一些好处，包括具有10-100倍的资源容量的潜力。虽然从计算机体系结构的角度来看是有益的，但分散体系结构改变了曾经指导现有网络、系统和应用程序的设计和优化的几个假设(例如，CPU存储位置、高CPU内存带宽、存储层次结构、数据局部性、故障模型等)。因此，利用非集中式体系结构的好处将需要重新设计传统系统和网络的体系结构。该项目旨在共同设计适用于非集中式数据中心的网络、存储和计算结构。在网络方面，该项目将设计超低延迟的机架内和机架间结构，包括包含高效拥塞控制、容错和调度机制的新网络软件堆栈。网络和存储结构的共同设计将导致用于分散存储的新的(分布式)内存和存储管理堆栈，以及一个资源管理器，该资源管理器在共享分散的存储和网络结构的多个应用程序之间提供基本的隔离、共享和弹性保证。最后，该项目将建立新的分布式编程框架，并重新设计现有应用程序，以便高效和正确地在分散的体系结构上运行。该项目将为围绕这一新兴计算模式的一些最困难和最重要的技术问题提供解决办法，并将主要通过教育和外联活动以及技术转让对社区产生广泛影响。这个项目产生的软件构件将被公开发布，以确保可重复性并促进后续研究。该项目还包括大量的教育内容，包括新课程和公开发行教材。最后，该项目将提供必要的推动力，通过指导研究生和博士后学者、年度研讨会和行业务虚会来建立跨学科研究社区，以弥合工业发展和学术研究之间的差距。

项目成果

CodedBulk: Inter-Datacenter Bulk Transfers using Network Coding

• 发表时间: 2021
• 作者: Shih-Hao Tseng;Saksham Agarwal;R. Agarwal;Hitesh Ballani;A. Tang

Understanding host network stack overheads

• DOI: 10.1145/3452296.3472888
• 发表时间: 2021-08
• 期刊: Proceedings of the 2021 ACM SIGCOMM 2021 Conference
• 作者: Qizhe Cai;Shubham Chaudhary;Midhul Vuppalapati;Jaehyun Hwang;R. Agarwal

Dagger: Towards Efficient RPCs in Cloud Microservices With Near-Memory Reconfigurable NICs

• DOI: 10.1109/lca.2020.3020064
• 发表时间: 2020-07-01
• 期刊: IEEE COMPUTER ARCHITECTURE LETTERS
• 影响因子: 2.3
• 作者: Lazarev, Nikita;Adit, Neil;Delimitrou, Christina
• 通讯作者: Delimitrou, Christina

Pancake: Frequency Smoothing for Encrypted Data Stores

Pancake：加密数据存储的频率平滑

• 发表时间: 2020
• 期刊: USENIX Security
• 作者: Grubbs, Paul;Khandelwal, Anurag;Lacharite, Marie-Sarah;Brown, Lloyd;Li, Lucy;Agarwal, Rachit;Ristenpart, Thomas
• 通讯作者: Ristenpart, Thomas

Sinan: Data-Driven Resource Management for Interactive Multi-tier Microservices

Sinan：交互式多层微服务的数据驱动资源管理

• 发表时间: 2020
• 期刊: Workshop on ML for Computer Architecture and Systems (MLArchSys
• 作者: Zhang, Yanqi;Hua, Weizhe;Zhou, Zhuangzhuang;Suh, Ed;Delimitrou, Christina
• 通讯作者: Delimitrou, Christina