Collaborative Research: SHF: Medium: Spatial Multi-Tenant Neural Acceleration for Next Generation Datacenters

人工智能的最新进展正在改变人类生活的许多方面，如电子商务，医疗，交通等。数据中心网络是现代在线服务的基础。随着全球从COVID-19疫情中复苏，社会对在线服务和机器学习的依赖日益增加。这种爆炸式的增长对计算中心的计算资源产生了巨大的需求。然而，今天的方法是极其昂贵和能源效率低下。事实上，如果目前的系统继续增长，到2040年，电子商务中心将占全球碳排放总量的14%。该项目旨在使用为机器学习工作负载量身定制的高级资源共享技术来应对这一挑战。特别是，该奖项使网络运营商能够最大限度地利用网络资源，同时为用户提供高质量的服务体验。这项工作旨在通过称为动态架构裂变的新范式探索多租户对机器学习加速的及时需求。在机器学习加速器方面，由于架构的刚性及其单租户性质，存在很大程度的利用不足。因此，迫切需要重新考虑自定义加速器的设计和在具有成本效益的资源利用取代不必要的资源克隆的数据中心中的采用。与微处理器的情况类似，多租户加速可以开辟一条补救资源复制和利用不足的途径。尽管如此，多租户并不是机器学习加速器设计中的主要因素，因为对更高速度的竞争，加速器在互联网中心的采用以及与加速器多租户相关的挑战。为此，该项目旨在探索空间多租户作为加速器设计的一个新维度，以解决数据中心的资源利用不足问题，并实现机器学习加速器的成本效益部署。这一新的维度将大大有助于降低云计算中心过度配置的斜率，为实现更绿色的云计算铺平道路。所提出的大规模深度学习的空间多租户加速可以大大提高下一代数据中心的成本效益。鉴于对深度学习服务的需求不断增长，以及训练和推理的碳足迹，这一提议将产生重大的社会经济和环境影响。研究人员也坚定地致力于扩大参与计算，并有全面的计划，使代表性不足的群体。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

Recent advances in Artificial Intelligence are transforming many aspects of human life such as e-commerce, medicine, transportation, and beyond. Datacenter networks are the foundation of modern online services. As the world is recovering from COVID-19, society is witnessing an increased reliance on online services and machine learning. This explosive growth has created an enormous demand for computation resources in datacenters. However, today's approaches are extremely costly and energy-inefficient. In fact, if the current systems continue to grow, datacenters will account for 14% of the total worldwide carbon emissions by 2040. This project aims to address this challenge using advanced resource-sharing techniques tailored for machine learning workloads. In particular, this award enables the network operators to maximize the utilization of network resources while achieving high quality of service experience for the users.This work sets out to explore the timely requirement of multi-tenancy for machine-learning acceleration through a new paradigm called dynamic architecture fission. There is a significant degree of underutilization when it comes to machine-learning accelerators that stem from the rigidity of architectures and their single-tenant nature. As such, there is an imminent need to rethink custom accelerator design and adoption in datacenters where cost-effective resource utilization replaces unnecessary resource cloning. Similar to the case of microprocessors, multi-tenant acceleration can open up a pathway that remedies resource replication and underutilization. Nonetheless, multi-tenancy has not been a primary factor in the design of machine-learning accelerators because of the race for higher speed, the recency of accelerator adoption in datacenters, and challenges associated with accelerator multi-tenancy. To that end, this project aims to explore spatial multi-tenancy as a new dimension in accelerator design to tackle resource underutilization in datacenters and bring forth cost-effective deployment of machine learning accelerators. This new dimension will significantly help reduce the slope of over-provisioning in datacenters to pave the way towards greener cloud computing. The proposed spatial multi-tenant acceleration of deep learning at scale can substantially improve the cost-effectiveness of next-generation datacenters. Given the increasing demand for deep-learning services and the carbon footprint of training and inference, this proposal will have a significant socioeconomic and environmental impact. The researchers are also strongly committed to broadening participation in computing and have comprehensive plans to engage the underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.