CNS Core: Small: Toward Globally-Optimal Resource Distribution and Computation Acceleration in Multi-Tenant and Heterogeneous Machine Learning Systems

在大规模深度学习（DL）和海量数据的时代，由于难以在集群或数据中心环境中的多台机器上调度来自许多用户的高动态、异构和竞争任务，现有的硬件系统难以有效地适应繁重和复杂的计算工作负载。该项目旨在开发一个“一键”需求感知和响应的软件系统，能够同时训练广泛的深度学习任务，使用新的资源管理架构，自动自适应地选择最有效的分布式训练/服务技术及其超参数，以在这种环境中实现多个任务的最佳整体效率。这个跨学科的项目在分布式系统设计、深度学习算法设计以及相关的工业应用和理论分析方面进行了创新，其重点如下：1：开发一个“机器学习感知”资源管理和多个同时运行的训练任务调度框架。2：开发服务、流和异构任务设置的资源管理和调度原则策略。3：优化内存资源用于训练大参数模型，通过开发整体方法来最大化受设备内存限制的计算吞吐量。还将对所提出的一些体系结构进行范围有限但严格且实用的理论分析。该项目满足了学术界和工业界的需求，并将对两者产生广泛的影响。它将提供易于使用的工具，减少设置时间和促进大规模实验，同时降低所需的成本，无论是以集群访问配额还是在云服务上花费的美元来衡量。对商业实践者的影响将更大，因为他们必须处理在许多用户之间共享的异构计算和网络资源，并且需要定期运行许多作业，从而将他们的生产力提高一个数量级或更多。该团队将在http://sailing-lab.wixsite.com/sailing-pmls上发布和/或开源代码，以使研究人员和从业者受益，分享他们的经验教训，倡导对机器学习（ML）系统问题进行更多的研究，并使高性能ML系统民主化，使非ML教育的软件开发人员和整个社会（如工业和制造业、医疗保健、生物学、社会科学和金融）都能使用它们。结果可能会产生催化作用。该团队将在各种顶级会议上发布结果，包括机器学习（NIPS， ICML），系统（OSDI， SOSP， USENIX）和数据挖掘（KDD， WWW）。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In the era of large-scale deep learning (DL) and massive data, existing hardware systems have struggled to effectively accommodate heavy and complex computing workload due to difficulties in scheduling highly dynamic, heterogeneous, and competing tasks from many users over many machines in a cluster or data-center environment. This project aims to develop a "1-click" demand-aware and responsive software system capable of simultaneously training a wide spectrum of DL tasks, using a new resource management architecture that automatically and adaptively chooses the most effective distributed training/serving techniques and their hyperparameters to achieve best overall efficiency of multiple tasks in such environment.This interdisciplinary project innovates in distributed systems design, DL algorithm design, and related industrial applications and theoretical analyses, with the following thrusts: 1: Develop a framework for "ML-aware" resource management and scheduling of multiple simultaneously running training tasks. 2: Develop principled strategies for resource management and scheduling for serving, streaming, and heterogeneous-task settings. 3: Optimize memory resources for training large-parameter models by developing holistic approaches to maximize computation throughput subject to device memory bounds. A limited-scope but rigorous and practical theoretical analysis of some of the proposed architectures will also be performed. This project addresses the needs from the academic and industrial communities and will have a broad impact on both. It will provide easy-to-use tools that reduce the time to set-up and facilitate large-scale experimentation, while reducing the required costs, whether measured in cluster access quotas or dollars spent on cloud services. The impact on commercial practitioners will be even greater, by improving their productivity by an order of magnitude or more, as they must contend with heterogeneous computing and network resources that are shared among many users as well as the need to run many jobs on a regular basis.The team will release and/or open-source the code at http://sailing-lab.wixsite.com/sailing-pmls to benefit researchers and practitioners, to share their lessons learned to advocate more research in machine learning (ML) systems problems, and also to democratize high-performance ML systems and make them accessible to non-ML-educated software developers and society at large, such as industrial and manufacturing, healthcare, biology, social science, and finance, where results may have a catalytic impact. The team will publish results at a variety of top tier conferences, including machine learning (NIPS, ICML), systems (OSDI, SOSP, USENIX), and data mining (KDD, WWW).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.