DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling
DRAS-CQSim: A Reinforcement Learning based Framework for HPC Cluster Scheduling
复制标题
DOI：
10.1016/j.simpa.2021.100077
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Yuping Fan;Z. Lan
中科院分区：
文献类型：
--
作者：
Yuping Fan;Z. Lan
几十年来，系统管理员一直致力于设计和调整集群调度策略，以提高高性能计算 (HPC) 系统的性能。然而，日益复杂的 HPC 系统与高度多样化的工作负载相结合，使得这种手动流程充满挑战、耗时且容易出错。我们提出了一种基于强化学习的 HPC 调度框架，名为 DRAS-CQSim，用于自动学习最优调度策略。 DRAS-CQSim封装了模拟环境、代理、超参数调整选项和不同的强化学习算法，使系统管理员能够快速获得定制的调度策略。
For decades, system administrators have been striving to design and tune cluster scheduling policies to improve the performance of high performance computing (HPC) systems. However, the increasingly complex HPC systems combined with highly diverse workloads make such manual process challenging, time-consuming, and error-prone. We present a reinforcement learning based HPC scheduling framework named DRAS-CQSim to automatically learn optimal scheduling policy. DRAS-CQSim encapsulates simulation environments, agents, hyperparameter tuning options, and different reinforcement learning algorithms, which allows the system administrators to quickly obtain customized scheduling policies.