Adaptive and Efficient Resource Allocation in Cloud Datacenters Using Actor-Critic Deep Reinforcement Learning
Adaptive and Efficient Resource Allocation in Cloud Datacenters Using Actor-Critic Deep Reinforcement Learning
复制标题
基于Actor-Critic深度强化学习的云数据中心自适应高效资源分配
DOI：
10.1109/tpds.2021.3132422
复制
发表时间：
2021
影响因子：
5.3
通讯作者：
Zheyi Chen;Jia Hu;G. Min;Chunbo Luo;T. El-Ghazawi
中科院分区：
文献类型：
--
作者：
Zheyi Chen;Jia Hu;G. Min;Chunbo Luo;T. El-Ghazawi
不断扩大的云数据中心规模要求自动资源调配能够最好地满足低延迟和高能效的要求。然而，由于系统状态的动态变化和用户需求的多样性，云中高效的资源分配面临着巨大的挑战。现有的云资源分配方案大多依赖于云系统的先验知识，不能有效地处理动态的云环境，这可能会导致过高的能量消耗和服务质量(Qos)的下降。针对这一问题，提出了一种基于Actor-Critic深度强化学习(DRL)的自适应高效云资源分配方案。首先，参与者将策略参数化(分配资源)，并根据批评者评估的分数(评估操作)选择操作(调度作业)。其次，利用梯度上升来更新资源分配策略，同时利用优势函数来降低策略梯度的方差，从而提高了训练效率。我们使用来自谷歌云数据中心的真实数据进行了广泛的模拟实验。实验结果表明，与改进的策略梯度DRL和五种经典的云资源分配方法相比，该方法可以在延迟和作业丢弃率方面获得更好的服务质量，并且具有更高的能量效率。
The ever-expanding scale of cloud datacenters necessitates automated resource provisioning to best meet the requirements of low latency and high energy-efficiency. However, due to the dynamic system states and various user demands, efficient resource allocation in cloud faces huge challenges. Most of the existing solutions for cloud resource allocation cannot effectively handle the dynamic cloud environments because they depend on the prior knowledge of a cloud system, which may lead to excessive energy consumption and degraded Quality-of-Service (QoS). To address this problem, we propose an adaptive and efficient cloud resource allocation scheme based on Actor-Critic Deep Reinforcement Learning (DRL). First, the actor parameterizes the policy (allocating resources) and chooses actions (scheduling jobs) based on the scores assessed by the critic (evaluating actions). Next, the resource allocation policy is updated by using gradient ascent while the variance of policy gradient is reduced with an advantage function, which improves the training efficiency of the proposed method. We conduct extensive simulation experiments using real-world data from Google cloud datacenters. The results show that our method can obtain superior QoS in terms of latency and job dismissing rate with enhanced energy-efficiency, compared to an advanced policy gradient DRL and five classic cloud resource allocation methods.