SHF:Small:Intelligent Management of Hybrid Workloads for Extreme Scale Computing

SHF:Small：超大规模计算混合工作负载的智能管理

• 批准号: 2413597
• 负责人: Zhiling Lan
• 金额: $ 50万
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2413597&HistoricalAwards=false
• 关键词: SHF; Small; Intelligent; Management; Hybrid

The high-performance computing (HPC) community is embracing artificial intelligence (AI) techniques for countless pursuits, from driving ground-breaking scientific discoveries to protecting our national security. As newly emerging machine learning and date-centric workloads proliferate in HPC, current workload-management systems cannot keep up with the significant challenges introduced by the diverse mix of applications co-running on heterogeneous systems. This project tackles the problem by developing new workload-management methods to catalyze the convergence of HPC, AI, and data analytics. It will develop fundamental improvements in HPC workload management to promote the use of large-scale supercomputers for emerging data-centric applications (HPC4AI). Meanwhile it will exploit advanced AI technologies, especially multi-objective reinforcement learning, to empower job scheduling and resource allocation in HPC (AI4HPC). The project aims to develop an intelligent workload-management framework named MINT in which distinctive computational resource requirements of hybrid workloads will be automatically identified and fulfilled to achieve extreme resource efficiency and satisfactory user experience. Key research thrusts are: understanding performance implications of diverse workloads on supercomputers via model-driven analysis; new intelligent multi-resource scheduling methods; smart resource-allocation strategies for minimal workload interference; and extensive evaluation of the proposed framework through trace-based simulation and testing. The deliverables include a new workload-management framework and open-source software releases for intelligent management of hybrid workloads on extreme-scale systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高性能计算（HPC）社区正在采用人工智能（AI）技术来实现无数追求，从推动开创性的科学发现到保护我们的国家安全。随着新兴的机器学习和以日期为中心的工作负载在HPC中扩散，当前的工作负载管理系统无法跟上各种应用程序在异构系统上共同运行的各种混合所带来的重大挑战。该项目通过开发新的工作负载管理方法来解决问题，以促进HPC，AI和数据分析的收敛性。它将在HPC工作负载管理方面进行基本改进，以促进用于新兴数据中心应用程序（HPC4AI）的大规模超级计算机的使用。同时，它将利用先进的AI技术，尤其是多目标增强学习，以增强HPC（AI4HPC）的工作计划和资源分配。 该项目旨在开发一个名为MINT的智能工作负载管理框架，其中将自动识别并实现混合工作负载的独特计算资源需求，以实现极端的资源效率和令人满意的用户体验。 关键的研究推力是：通过模型驱动分析，了解超级计算机对各种工作量的绩效含义；新的智能多资源调度方法；最小工作负载干扰的智能资源分配策略；并通过基于痕量的仿真和测试对拟议框架进行广泛评估。可交付成果包括一个新的工作负载管理框架和开源软件发布，以智能管理极端尺度系统的混合工作负载。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响审查标准通过评估来进行评估的。