Resource allocation for MEC system with multi-users resource competition based on deep reinforcement learning approach
Resource allocation for MEC system with multi-users resource competition based on deep reinforcement learning approach
复制标题
DOI：
10.1016/j.comnet.2022.109181
复制
发表时间：
2022-08-01
影响因子：
5.6
通讯作者：
Li，Xianxian
中科院分区：
文献类型：
--
作者：
Qu，Bin;Bai，Yan;Li，Xianxian
移动的边缘计算（MEC）是5G时代移动的设备降低计算延迟和能耗的有效计算范式。然而，在多用户资源竞争环境中，边缘服务器的收入驱动行为会导致一些用户增加延迟或任务失败。考虑到这种情况，我们将计算卸载的成功率作为边缘服务器的信任值，并从用户的角度构建系统模型，将延迟和能耗作为联合优化的多目标任务。在优化目标中，我们考虑了三个因素：卸载延迟，能量消耗和排队延迟。同时最小化能耗和时延是一个矛盾问题。因此，我们在优先考虑卸载成功率（减少延迟）的情况下，基于尽可能减少能耗的原则来解决问题。此外，我们建立了一个马尔可夫决策问题（MDP）的多因素奖励值的问题，并把信任值作为一个状态的系统。最后，我们使用扩展的深度确定性策略梯度（DDPG）算法（一种具有多目标奖励的DDPG算法）来解决这个问题。实验结果表明，我们提出的方案可以更好地减少延迟和能量消耗的计算卸载的移动的用户（MU）显着优于基线计划。在计算资源紧张的环境中，我们所提出的方案的优势更加明显。
Mobile edge computing (MEC) is an effective computing paradigm for mobile devices in the 5G era to reduce computing delay and energy consumption. However, in a multi-user resource competition environment, the revenue-driven behavior of edge servers will cause some users to increase delays or fail tasks. Considering this situation, we take the success rate of computation offloading as the trust value of the edge server, and build a system model from the user’s perspective, taking delay and energy consumption as the multi-objective task of joint optimization. In the optimization goal, we consider three factors: offloading delay, energy consumption, and queuing delay. Simultaneously minimizing energy consumption and delay is a contradiction problem. Therefore, we solve the problem based on the principle of reducing energy consumption as much as possible when the offload success rate (decreasing delay) is prioritized. Further, we build the problem as a Markov decision problem (MDP) with multi-factor reward value, and treat the trust value as a state of the system. Finally, we use an extended deep deterministic policy gradient (DDPG) algorithm (a DDPG algorithm with multi-objective reward) to work around this problem. Experimental results show that our proposed scheme can better reduce the delay and energy consumption in computation offloading of mobile users (MUs) significantly better than the baseline schemes. The advantages of our proposed scheme are more obvious in an environment where computing resources are tight.