NeTS: CSR: Large: Collaborative Research: Co-Design of Network, Storage and Computation Fabrics for Disaggregated Datacenters

NeTS：CSR：大型：协作研究：分解数据中心的网络、存储和计算结构的协同设计

• 批准号: 1704941
• 负责人: Sylvia Ratnasamy
• 金额: $ 90万
• 依托单位: International Computer Science Institute
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-15 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1704941&HistoricalAwards=false
• 关键词: NeTS; CSR; Large; Collaborative; Research

Traditional datacenters are built using servers, each of which tightly integrates a small amount of CPU (central processing unit), memory and storage onto a single motherboard. However, the end of Dennard's scaling and the slowdown of Moore's Law has led to surfacing of several fundamental limitations of such server-centric architectures (e.g., the memory-capacity wall making CPU-memory co-location unsustainable). Consequently, a new computing paradigm is emerging -- a disaggregated architecture, where each resource type is built as a standalone 'blade' and a network fabric interconnects the resource blades within and across racks. The computer architecture community has established a number of benefits of such disaggregated architectures, including the potential to have 10-100x larger resource capacity. While beneficial from the computer architecture perspective, disaggregated architectures alter several assumptions that once guided the design and optimization of existing networks, systems and applications (e.g., CPU-storage colocation, high CPU-memory bandwidth, storage hierarchy, data locality, failure models, etc.). Capitalizing on the benefits of disaggregated architectures will thus require re-architecting legacy systems and networks. This project aims to co-design the network, storage and compute fabrics for disaggregated datacenters.On the network front, the project will design ultra-low latency intra-rack and inter-rack fabrics including a new network software stack that incorporates efficient congestion control, failure tolerance and scheduling mechanisms. The co-design of network and storage fabrics will lead to new (distributed) memory and storage management stacks for disaggregated storage, and a resource manager that provides essential isolation, sharing and elasticity guarantees across multiple applications sharing disaggregated storage and network fabrics. Finally, the project will build new distributed programming frameworks and re-architect existing applications to efficiently and correctly operate on disaggregated architectures. This project will provide solutions to some of the most difficult and important technical questions surrounding this emerging computing paradigm and will have broad community impact primarily through educational and outreach activities, and technology transfer. Software artifacts resulting from this project will be publicly released to ensure repeatability and to foster follow up research. The project also has a substantial educational component including new courses and public release of teaching materials. Finally, the project will provide the necessary thrust to build an inter-disciplinary research community via mentoring of graduate students and postdoctoral scholars, yearly workshops and industry retreats to bridge the gap between industrial development and academic research.

传统的数据中心是使用服务器构建的，每个服务器都将少量CPU（中央处理器），内存和存储紧密集成到单个主板上。然而，Dennard扩展的结束和摩尔定律的放缓已经导致这种以服务器为中心的架构的几个基本限制（例如，内存容量墙使得CPU-内存协同定位不可持续）。因此，一种新的计算模式正在出现--一种分散式架构，其中每种资源类型都被构建为独立的“刀片”，并且网络结构将机架内和机架之间的资源刀片互连。计算机架构社区已经确定了这种分解架构的许多好处，包括具有10- 100倍的资源容量的潜力。虽然从计算机体系结构的角度来看是有益的，但分解的体系结构改变了曾经指导现有网络、系统和应用程序的设计和优化的几个假设（例如，CPU存储托管、高CPU内存带宽、存储层次结构、数据局部性、故障模型等）。因此，要充分利用分散式架构的好处，就需要重新设计遗留系统和网络。 该项目旨在为分散式数据中心共同设计网络、存储和计算结构。在网络方面，该项目将设计超低延迟的机架内和机架间结构，包括一个新的网络软件栈，该软件栈包含有效的拥塞控制、容错和调度机制。网络和存储结构的共同设计将导致新的（分布式）内存和存储管理堆栈，用于分散存储，以及资源管理器，为共享分散存储和网络结构的多个应用程序提供必要的隔离，共享和弹性保证。最后，该项目将构建新的分布式编程框架，并重新构建现有的应用程序，以有效和正确地在分散的架构上运行。该项目将为围绕这一新兴计算模式的一些最困难和最重要的技术问题提供解决方案，并将主要通过教育和外联活动以及技术转让产生广泛的社区影响。该项目产生的软件工件将公开发布，以确保可重复性并促进后续研究。该项目还具有重要的教育组成部分，包括新课程和公开发布教材。最后，该项目将提供必要的推动力，通过指导研究生和博士后学者、年度讲习班和行业务虚会，建立一个跨学科的研究社区，以弥合工业发展和学术研究之间的差距。

项目成果

Monotasks: Architecting for Performance Clarity in Data Analytics Frameworks

单任务：数据分析框架中性能清晰度的架构

• DOI: 10.1145/3132747.3132766
• 发表时间: 2017
• 期刊: SOSP 2017
• 作者: Ousterhout, Kay;Canel, Christopher;Ratnasamy, Sylvia;Shenker, Scott
• 通讯作者: Shenker, Scott

Efficient Scheduling Policies for Microsecond-Scale Tasks

微秒级任务的高效调度策略

• 发表时间: 2022
• 期刊: USENIX Symposium on Networked Systems Design and Implementation
• 作者: Sarah McClure, Amy Ousterhout

Datacenter congestion control: identifying what is essential and making it practical

• DOI: 10.1145/3371927.3371932
• 发表时间: 2019-11
• 期刊: Comput. Commun. Rev.
• 作者: Aisha Mushtaq;R. Mittal;J. McCauley;Mohammad Alizadeh;Sylvia Ratnasamy;S. Shenker

Remote Memory Calls

• DOI: 10.1145/3422604.3425923
• 发表时间: 2020-11
• 期刊: Proceedings of the 19th ACM Workshop on Hot Topics in Networks
• 作者: Emmanuel Amaro;Zhihong Luo;Amy Ousterhout;A. Krishnamurthy;Aurojit Panda;Sylvia Ratnasamy;S. Shenker

Bertha: Tunneling through the Network API

• DOI: 10.1145/3422604.3425927
• 发表时间: 2020-11
• 期刊: Proceedings of the 19th ACM Workshop on Hot Topics in Networks
• 作者: Akshay Narayan;Aurojit Panda;Mohammad Alizadeh;H. Balakrishnan;A. Krishnamurthy;S. Shenker