Collaborative Research: SHF: Medium: Spatial Multi-Tenant Neural Acceleration for Next Generation Datacenters

人工智能的最新进展正在改变人类生活的许多方面，如电子商务、医药、交通等。数据中心网络是现代在线服务的基础。随着世界从COVID-19中恢复过来，社会越来越依赖在线服务和机器学习。这种爆炸性的增长对数据中心的计算资源产生了巨大的需求。然而，目前的方法非常昂贵且能源效率低下。事实上，如果目前的系统继续增长，到2040年，数据中心将占全球碳排放总量的14%。该项目旨在使用为机器学习工作负载量身定制的先进资源共享技术来解决这一挑战。特别是，该奖项使网络运营商能够最大限度地利用网络资源，同时为用户提供高质量的服务体验。这项工作旨在通过一种称为动态架构裂变的新范式探索多租户对机器学习加速的及时要求。当涉及到机器学习加速器时，由于架构的刚性及其单租户性质，存在很大程度的未充分利用。因此，迫切需要重新考虑数据中心中定制加速器的设计和采用，在这些数据中心中，具有成本效益的资源利用取代了不必要的资源克隆。与微处理器的情况类似，多租户加速可以开辟一条途径来补救资源复制和利用不足。尽管如此，多租户并不是机器学习加速器设计中的主要因素，因为对更高速度的竞争、数据中心中最近采用的加速器以及与加速器多租户相关的挑战。为此，本项目旨在探索空间多租户作为加速器设计的新维度，以解决数据中心资源利用不足的问题，并提出具有成本效益的机器学习加速器部署。这个新的维度将极大地帮助减少数据中心过度配置的倾斜，为更绿色的云计算铺平道路。提出的空间多租户深度学习的大规模加速可以大大提高下一代数据中心的成本效益。鉴于对深度学习服务的需求不断增长以及训练和推理的碳足迹，该提案将产生重大的社会经济和环境影响。研究人员还坚定地致力于扩大对计算机的参与，并制定了全面的计划，以吸引代表性不足的群体。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advances in Artificial Intelligence are transforming many aspects of human life such as e-commerce, medicine, transportation, and beyond. Datacenter networks are the foundation of modern online services. As the world is recovering from COVID-19, society is witnessing an increased reliance on online services and machine learning. This explosive growth has created an enormous demand for computation resources in datacenters. However, today's approaches are extremely costly and energy-inefficient. In fact, if the current systems continue to grow, datacenters will account for 14% of the total worldwide carbon emissions by 2040. This project aims to address this challenge using advanced resource-sharing techniques tailored for machine learning workloads. In particular, this award enables the network operators to maximize the utilization of network resources while achieving high quality of service experience for the users.This work sets out to explore the timely requirement of multi-tenancy for machine-learning acceleration through a new paradigm called dynamic architecture fission. There is a significant degree of underutilization when it comes to machine-learning accelerators that stem from the rigidity of architectures and their single-tenant nature. As such, there is an imminent need to rethink custom accelerator design and adoption in datacenters where cost-effective resource utilization replaces unnecessary resource cloning. Similar to the case of microprocessors, multi-tenant acceleration can open up a pathway that remedies resource replication and underutilization. Nonetheless, multi-tenancy has not been a primary factor in the design of machine-learning accelerators because of the race for higher speed, the recency of accelerator adoption in datacenters, and challenges associated with accelerator multi-tenancy. To that end, this project aims to explore spatial multi-tenancy as a new dimension in accelerator design to tackle resource underutilization in datacenters and bring forth cost-effective deployment of machine learning accelerators. This new dimension will significantly help reduce the slope of over-provisioning in datacenters to pave the way towards greener cloud computing. The proposed spatial multi-tenant acceleration of deep learning at scale can substantially improve the cost-effectiveness of next-generation datacenters. Given the increasing demand for deep-learning services and the carbon footprint of training and inference, this proposal will have a significant socioeconomic and environmental impact. The researchers are also strongly committed to broadening participation in computing and have comprehensive plans to engage the underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.