GreenDRL: managing green datacenters using deep reinforcement learning
GreenDRL: managing green datacenters using deep reinforcement learning
复制标题
DOI：
10.1145/3542929.3563501
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Kuo Zhang;Peijian Wang;Ning Gu;Thu D. Nguyen
中科院分区：
文献类型：
--
作者：
Kuo Zhang;Peijian Wang;Ning Gu;Thu D. Nguyen
管理数据中心以最大程度地提高效率和可持续性是一个复杂且具有挑战性的问题。在这项工作中，我们探讨了深入强化学习（RL）来管理“绿色”数据中心的使用，为设计有效的管理系统提供了强大的方法，以说明特定的工作负载，数据中心和环境特征。我们设计和评估Greendrl，该系统将深度RL代理与简单的启发式方法结合在一起，以管理工作量，能耗和冷却，并在现场生成可再生能源的存在下，以最大程度地减少棕色的能耗和成本。我们的设计解决了一些重要的挑战，包括适应性，鲁棒性和在包括巨大状态/行动空间和多个随机过程的环境中的有效学习。评估结果（使用模拟）表明，Greendrl能够学习重要的原则，例如延迟可延期的工作来利用可变的可再生能源（太阳能）能源，并避免使用电力密集型冷却设置，即使以牺牲一些可再生能源为准未使用。在最多可延期12小时的工作量的环境中，与FIFO基线方法相比，GREENDRL可以减少不同太阳能产生和温度特性的数天的电网消耗32--54％。 Greendrl还匹配或胜过使用线性编程以及Oracular Future知识来管理工作负载和服务器能源消耗的管理方法，但将冷却系统的管理保留到一个单独的（且独立的）控制器。总体而言，我们的工作表明，DEEP RL是为绿色数据中心构建有效管理系统的有前途的技术。
Managing datacenters to maximize efficiency and sustain-ability is a complex and challenging problem. In this work, we explore the use of deep reinforcement learning (RL) to manage "green" datacenters, bringing a robust approach for designing efficient management systems that account for specific workload, datacenter, and environmental characteristics. We design and evaluate GreenDRL, a system that combines a deep RL agent with simple heuristics to manage workload, energy consumption, and cooling in the presence of onsite generation of renewable energy to minimize brown energy consumption and cost. Our design addresses several important challenges, including adaptability, robustness, and effective learning in an environment comprising an enormous state/action space and multiple stochastic processes. Evaluation results (using simulation) show that GreenDRL is able to learn important principles such as delaying deferrable jobs to leverage variable generation of renewable (solar) energy, and avoiding the use of power-intensive cooling settings even at the expense of leaving some renewable energy unused. In an environment where a fraction of the workload is deferrable by up to 12 hours, GreenDRL can reduce grid electricity consumption for days with different solar energy generation and temperature characteristics by 32--54% compared to a FIFO baseline approach. GreenDRL also matches or outperforms a management approach that uses linear programming together with oracular future knowledge to manage workload and server energy consumption, but leaves the management of the cooling system to a separate (and independent) controller. Overall, our work shows that deep RL is a promising technique for building efficient management systems for green datacenters.