Collaborative Research: CNS Core: Small: Optimizing Large-Scale Heterogeneous ML Platforms

合作研究：CNS Core：小型：优化大规模异构机器学习平台

• 批准号: 2146814
• 负责人: Adam Wierman
• 金额: $ 25万
• 依托单位: California Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2146814&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Small

Large-scale artificial intelligence and machine learning (AI/ML) platforms are playing a vital role in the current data revolution. To minimize efforts from users, an end-to-end solution is desired to deploy complex workflow over possibly heterogeneous computing clusters. However, the scheduling and resource management problems behind such “push-button” deployment are challenging. If left unsolved, these costly systems will be severely under-utilized, leading to unnecessary electricity consumption and greenhouse gas emissions. This project will develop efficient resource allocation policies for distributed, large-scale AI/ML systems to tackle the challenges. Specifically, this project will accelerate and parallelize the large-scale optimization and inference tasks that dominate workloads in AI/ML platforms via distributed optimization that provides fault tolerance and robustness to stragglers in heterogeneous settings. Built upon the distributed optimization, the project will further schedule AI/ML workflows with precedence constraints among sub-tasks. Finally, heterogeneous resources are allocated among jobs fairly and efficiently in the case where the resources being allocated are exchangeable, which is key for AI/ML platforms with graphic processing units (GPUs) and other accelerators. The project will provide new fundamental algorithms for scheduling and resource allocation in AI/ML platforms used across academia and industry. The algorithmic ideas will be developed in the context of core, classical models and so will apply more broadly than AI/ML platforms, e.g., to networking, storage, supply chain management, and beyond. The project will seek to broaden the participation of underrepresented groups in Science, Technology, Engineering and Mathematics by planned activities including the development of accelerated mathematics programs for middle school students, summer programs for middle-school and high-school students, and summer research programs for undergraduate students.The project will make its software artifacts, datasets, and research results available to the research community on the project website at https://adamwierman.com/optimizing-large-scale-heterogeneous-ml-platforms/ Artifacts will be maintained for a minimum of 10 years.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

大规模人工智能和机器学习(AI/ML)平台在当前的数据革命中发挥着至关重要的作用。为了最大限度地减少用户的工作量，需要一种端到端的解决方案来在可能的异类计算集群上部署复杂的工作流。然而，这种“按键”部署背后的调度和资源管理问题是具有挑战性的。如果不加以解决，这些昂贵的系统将严重得不到充分利用，导致不必要的电力消耗和温室气体排放。该项目将为分布式、大规模的AI/ML系统制定有效的资源分配策略，以应对这些挑战。具体地说，该项目将通过为异类环境中的落后者提供容错和健壮性的分布式优化，加速和并行化AI/ML平台中主导工作负载的大规模优化和推理任务。在分布式优化的基础上，该项目将进一步调度具有子任务之间优先约束的AI/ML工作流。最后，在分配的资源是可交换的情况下，在作业之间公平高效地分配异质资源，这是具有图形处理器(GPU)和其他加速器的AI/ML平台的关键。该项目将为学术界和工业界使用的AI/ML平台中的调度和资源分配提供新的基本算法。算法思想将在核心、经典模型的背景下开发，因此将比AI/ML平台应用更广泛，例如，网络、存储、供应链管理等。该项目将寻求通过有计划的活动扩大科学、技术、工程和数学领域未被充分代表的群体的参与，包括为中学生开发加速数学课程，为中学生和高中生开发暑期课程，为本科生提供暑期研究课程。该项目将使其软件制品、数据集、该奖项反映了https://adamwierman.com/optimizing-large-scale-heterogeneous-ml-platforms/的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

The Online Pause and Resume Problem: Optimal Algorithms and An Application to Carbon-Aware Load Shifting

在线暂停和恢复问题：最优算法和碳感知负载转移的应用

• DOI: 10.1145/3626776
• 发表时间: 2023
• 期刊: Proceedings of the ACM on Measurement and Analysis of Computing Systems
• 作者: Lechowicz, Adam;Christianson, Nicolas;Zuo, Jinhang;Bashir, Noman;Hajiesmaili, Mohammad;Wierman, Adam;Shenoy, Prashant
• 通讯作者: Shenoy, Prashant

The Online Knapsack Problem with Departures

出发时的在线背包问题

• DOI: 10.1145/3570618
• 发表时间: 2022
• 期刊: Proceedings of the ACM on Measurement and Analysis of Computing Systems
• 作者: Sun, Bo;Yang, Lin;Hajiesmaili, Mohammad;Wierman, Adam;Lui, John C.;Towsley, Don;Tsang, Danny H.K.
• 通讯作者: Tsang, Danny H.K.

Smoothed Online Optimization with Unreliable Predictions

• DOI: 10.1145/3579442
• 发表时间: 2022-02
• 期刊: Proceedings of the ACM on Measurement and Analysis of Computing Systems
• 作者: Daan Rutten;Nicolas H. Christianson;Debankur Mukherjee;A. Wierman

Robustness and Consistency in Linear Quadratic Control with Untrusted Predictions

具有不可信预测的线性二次控制的鲁棒性和一致性

• DOI: 10.1145/3508038
• 发表时间: 2022
• 期刊: Proceedings of the ACM on Measurement and Analysis of Computing Systems
• 作者: Li, Tongxin;Yang, Ruixiao;Qu, Guannan;Shi, Guanya;Yu, Chenkai;Wierman, Adam;Low, Steven
• 通讯作者: Low, Steven

Chasing convex bodies and functions with black-box advice

用黑盒建议追逐凸体和函数

• 发表时间: 2022
• 期刊: COLT
• 作者: Nicolas Christianson, Tinashe Handina