Collaborative Research: SHF: Medium: Spatial Multi-Tenant Neural Acceleration for Next Generation Datacenters

人工智能的最新进展正在改变人类生活的许多方面，如电子商务、医疗、交通等。数据中心网络是现代在线服务的基础。随着世界从新冠肺炎中复苏，社会对在线服务和机器学习的依赖也在增加。这种爆炸性的增长对数据中心的计算资源产生了巨大的需求。然而，今天的方法极其昂贵，而且能源效率低下。事实上，如果目前的系统继续增长，到2040年，数据中心将占全球碳排放总量的14%。该项目旨在使用为机器学习工作负载量身定做的高级资源共享技术来应对这一挑战。特别是，该奖项使网络运营商能够最大限度地利用网络资源，同时为用户实现高质量的服务体验。这项工作旨在通过一种名为动态架构裂变的新范式来探索多租户对机器学习加速的及时需求。当涉及到机器学习加速器时，由于架构的僵化及其单租户性质，机器学习加速器的利用率很低。因此，迫切需要重新考虑数据中心中的定制加速器设计和采用，在数据中心中，经济高效的资源利用取代了不必要的资源克隆。与微处理器的情况类似，多租户加速可以开辟一条补救资源复制和未充分利用的途径。尽管如此，多租户并不是机器学习加速器设计的主要因素，这是因为对更高速度的竞争、数据中心采用加速器的新近以及与加速器多租户相关的挑战。为此，该项目旨在探索空间多租户作为加速器设计的新维度，以解决数据中心资源利用不足的问题，并提出具有成本效益的机器学习加速器部署。这一新维度将有助于显著降低数据中心过度配置的斜率，为更环保的云计算铺平道路。提出的大规模深度学习空间多租户加速可以大幅提高下一代数据中心的成本效益。鉴于对深度学习服务的需求日益增加，以及培训和推理的碳足迹，这一提议将产生重大的社会经济和环境影响。研究人员还坚定地致力于扩大对计算的参与，并制定了全面的计划，以吸引代表不足的群体。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in Artificial Intelligence are transforming many aspects of human life such as e-commerce, medicine, transportation, and beyond. Datacenter networks are the foundation of modern online services. As the world is recovering from COVID-19, society is witnessing an increased reliance on online services and machine learning. This explosive growth has created an enormous demand for computation resources in datacenters. However, today's approaches are extremely costly and energy-inefficient. In fact, if the current systems continue to grow, datacenters will account for 14% of the total worldwide carbon emissions by 2040. This project aims to address this challenge using advanced resource-sharing techniques tailored for machine learning workloads. In particular, this award enables the network operators to maximize the utilization of network resources while achieving high quality of service experience for the users.This work sets out to explore the timely requirement of multi-tenancy for machine-learning acceleration through a new paradigm called dynamic architecture fission. There is a significant degree of underutilization when it comes to machine-learning accelerators that stem from the rigidity of architectures and their single-tenant nature. As such, there is an imminent need to rethink custom accelerator design and adoption in datacenters where cost-effective resource utilization replaces unnecessary resource cloning. Similar to the case of microprocessors, multi-tenant acceleration can open up a pathway that remedies resource replication and underutilization. Nonetheless, multi-tenancy has not been a primary factor in the design of machine-learning accelerators because of the race for higher speed, the recency of accelerator adoption in datacenters, and challenges associated with accelerator multi-tenancy. To that end, this project aims to explore spatial multi-tenancy as a new dimension in accelerator design to tackle resource underutilization in datacenters and bring forth cost-effective deployment of machine learning accelerators. This new dimension will significantly help reduce the slope of over-provisioning in datacenters to pave the way towards greener cloud computing. The proposed spatial multi-tenant acceleration of deep learning at scale can substantially improve the cost-effectiveness of next-generation datacenters. Given the increasing demand for deep-learning services and the carbon footprint of training and inference, this proposal will have a significant socioeconomic and environmental impact. The researchers are also strongly committed to broadening participation in computing and have comprehensive plans to engage the underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.