Channel Access and Power Control for Energy-Efficient Delay-Aware Heterogeneous Cellular Networks for Smart Grid Communications Using Deep Reinforcement Learning
Channel Access and Power Control for Energy-Efficient Delay-Aware Heterogeneous Cellular Networks for Smart Grid Communications Using Deep Reinforcement Learning
复制标题
DOI：
10.1109/access.2019.2939827
复制
发表时间：
2019-01-01
期刊：
影响因子：
3.9
通讯作者：
Imran, Muhammad Ali
中科院分区：
文献类型：
--
作者：
Asuhaimi, Fauzun Abdullah;Bu, Shengrong;Imran, Muhammad Ali
基于长期演进（LTE）标准的蜂窝技术由于其高可用性和可扩展性而成为智能电网邻域网络的优选选择。然而，蜂窝网络和智能电网通信的集成提出了一个重大挑战，因为实时智能电网数据的同时传输可能会导致无线电接入网络（RAN）拥堵。已经提出了异构蜂窝网络（HetNet）来改善LTE的性能，因为HetNet可以通过将接入尝试从宏小区卸载到小小区来减轻RAN拥塞。在本文中，我们研究的能量效率和延迟问题，在HetNets传输智能电网数据具有不同的延迟要求。我们提出了一个分布式的信道接入和功率控制方案，并开发了一个基于学习的方法相量测量单元（PMU）成功地传输数据，通过考虑干扰和信号干扰噪声比（SINR）的约束。特别是，我们利用基于深度强化学习（DRL）的方法来训练PMU学习最佳策略，该策略可以在不了解系统动态的情况下最大化成功传输的奖励。仿真结果表明，DRL策略在不需要预先知道系统动态的情况下获得了良好的性能，在不同的正常比、最小SINR要求和小区用户数下，性能优于Gittin索引策略。
Cellular technology with long-term evolution (LTE)-based standards is a preferable choice for smart grid neighborhood area networks due to its high availability and scalability. However, the integration of cellular networks and smart grid communications puts forth a significant challenge due to the simultaneous transmission of real-time smart grid data which could cause radio access network (RAN) congestions. Heterogeneous cellular networks (HetNets) have been proposed to improve the performance of LTE because HetNets can alleviate RAN congestions by off-loading access attempts from a macrocell to small cells. In this paper, we study energy efficiency and delay problems in HetNets for transmitting smart grid data with different delay requirements. We propose a distributed channel access and power control scheme, and develop a learning-based approach for the phasor measurement units (PMUs) to transmit data successfully by considering interference and signal-to-interference-plus-noise ratio (SINR) constraints. In particular, we exploit a deep reinforcement learning(DRL)-based method to train the PMUs to learn an optimal policy that maximizes the earned reward of successful transmissions without having knowledge on the system dynamics. Results show that the DRL approach obtains good performance without knowing the system dynamic beforehand and outperforms the Gittin index policy in different normal ratios, minimum SINR requirements and number of users in the cell.