A Deep Reinforcement Learning Based Offloading Game in Edge Computing
A Deep Reinforcement Learning Based Offloading Game in Edge Computing
复制标题
边缘计算中基于深度强化学习的卸载游戏
DOI：
10.1109/tc.2020.2969148
复制
发表时间：
2020-06-01
影响因子：
3.7
通讯作者：
Zhang, Jiang
中科院分区：
文献类型：
--
作者：
Zhan, Yufeng;Guo, Song;Zhang, Jiang
边缘计算是在普适无线接入网边缘靠近用户提供强大计算能力的新范式。边缘计算的一个关键研究挑战是设计一种有效的卸载策略，以确定哪些任务可以在有限的资源下卸载到边缘服务器上。尽管许多研究努力试图解决这一挑战，但它们需要集中控制，这是不切实际的，因为用户是理性的个体，他们的利益最大化。在本文中，我们研究设计一种分散的计算卸载算法，使用户可以独立选择他们的卸载决策。在算法设计中应用了博弈论。与现有的工作不同，我们解决了用户可能拒绝暴露他们关于网络带宽和偏好的信息的挑战。因此，它要求我们的解决方案应该在没有这些知识的情况下做出卸载决策。我们将该问题表述为部分可观察的马尔可夫决策过程（POMDP），该过程通过基于策略梯度深度强化学习（DRL）的方法来解决。大量的仿真结果表明，我们的方案明显优于现有的解决方案。
Edge computing is a new paradigm to provide strong computing capability at the edge of pervasive radio access networks close to users. A critical research challenge of edge computing is to design an efficient offloading strategy to decide which tasks can be offloaded to edge servers with limited resources. Although many research efforts attempt to address this challenge, they need centralized control, which is not practical because users are rational individuals with interests to maximize their benefits. In this article, we study to design a decentralized algorithm for computation offloading, so that users can independently choose their offloading decisions. Game theory has been applied in the algorithm design. Different from existing work, we address the challenge that users may refuse to expose their information about network bandwidth and preference. Therefore, it requires that our solution should make the offloading decision without such knowledge. We formulate the problem as a partially observable Markov decision process (POMDP), which is solved by a policy gradient deep reinforcement learning (DRL) based approach. Extensive simulation results show that our proposal significantly outperforms existing solutions.