Market-Based Model in CR-IoT: A Q-Probabilistic Multi-Agent Reinforcement Learning Approach
Market-Based Model in CR-IoT: A Q-Probabilistic Multi-Agent Reinforcement Learning Approach
复制标题
CR-IoT 中基于市场的模型：Q 概率多代理强化学习方法
DOI：
10.1109/tccn.2019.2950242
复制
发表时间：
2020-03-01
影响因子：
8.6
通讯作者：
Guizani, Mohsen
中科院分区：
文献类型：
--
作者：
Wang, Dan;Zhang, Wei;Guizani, Mohsen
不断增长的城市人口和相应的物质需求给城市带来了前所未有的负担。为了保证市民获得更好的QoS，智慧城市利用了认知无线电物联网（CR-IoT）等新兴技术。然而，资源分配对CR-IoT来说是一个巨大的挑战，主要是因为设备和用户数量非常多。通常，拍卖理论和博弈论被用来克服这一挑战。在本文中，我们提出了一个多智能体强化学习（MARL）算法学习寡头垄断市场模型中的最优资源分配策略。首先，我们建立了一个多智能体场景，主要用户（PU）作为卖家和次要用户（SU）作为买家。然后，我们提出了Q-概率多智能体学习（QPML），并将其应用于市场中的资源分配。在多智能体学习过程中，PU和SU学习策略以最大化其利益并提高频谱利用率。QPML的性能进行了比较与学习自动化（LA）通过模拟。实验结果表明，我们的方法优于其他方法，表现良好。
The ever-increasing urban population and the corresponding material demands have brought unprecedented burdens to cities. To guarantee better QoS for citizens, smart cities leverage emerging technologies such as the Cognitive Radio Internet of Things (CR-IoT). However, resource allocation is a great challenge for CR-IoT, mainly because of the extremely numerous devices and users. Generally, the auction theory and game theory are applied to overcome the challenge. In this paper, we propose a multi-agent reinforcement learning (MARL) algorithm to learn the optimal resource allocation strategy in the oligopoly market model. Firstly, we model a multi-agent scenario with the primary users (PUs) as sellers and secondary users (SUs) as buyers. Then, we propose the Q-probabilistic multi-agent learning (QPML) and apply it to allocate resources in the market. In the multi-agent learning process, the PUs and SUs learn strategies to maximize their benefits and improve spectrum utilization. The performance of QPML is compared with Learning Automation (LA) through simulations. The experimental results show that our approach outperforms other approaches and performs well.