Optimal Resource Allocation for Reconfigurable Intelligent Surface Assisted Dynamic Wireless Network via Online Reinforcement Learning
Optimal Resource Allocation for Reconfigurable Intelligent Surface Assisted Dynamic Wireless Network via Online Reinforcement Learning
复制标题
DOI：
10.1109/seconworkshops56311.2022.9926399
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Yuzhu Zhang;Hao Xu
中科院分区：
文献类型：
--
作者：
Yuzhu Zhang;Hao Xu
研究了具有不确定时变信道的可重构智能曲面（RIS）辅助动态无线网络的资源优化分配问题。近年来，RIS被认为是在不增加功耗的情况下提高动态无线网络质量（如最大化频谱效率等）最有前途的技术之一。然而，传统的资源分配算法不能直接用于RIS辅助无线网络，特别是当基站（BS）、RIS和用户（ue）之间的无线信道不确定且时变时。为此，本文提出了一种基于在线强化学习的资源优化分配算法。首先，将具有动态无线信道的ris辅助无线通信网络表示为状态空间模型；然后，将资源最优分配问题表述为用户发射功率和RIS相移的有限视界联合最优控制问题。其次，由于无线信道具有时变和不确定性，结合神经网络（NN），提出了一种新的在线强化学习技术，即Actor-Critic设计，以实时学习最优资源分配策略。最后，通过数值模拟验证了该方案的有效性。
This paper investigates the problem of optimal resource allocation for reconfigurable intelligent surface (RIS) assisted dynamic wireless networks with uncertain time-varying wireless channels. Recently, RIS has been considered as one of the most promising techniques for enhancing dynamic wireless network quality, e.g. maximizing spectrum efficiency, etc., without increasing power consumption. However, conventional resource allocation algorithms cannot be directly utilized for RIS-assisted wireless networks especially when the wireless channels among base station (BS), RIS, and users (UEs) are uncertain and time varying. Hence, a novel online reinforcement learning based optimal resource allocation algorithm has been developed in this paper. Firstly, the RIS-assisted wireless communication network with dynamic wireless channels has been represented as a state-space model. Then, the optimal resource allocation problem can be formulated as a finite-horizon joint optimal control of users' transmit powers and RIS phase shifts problem. Next, since the wireless channel is time-varying and uncertain, a novel online reinforcement learning technique, i.e. Actor-Critic design, has been developed along with neural networks (NN) to learn the optimal resource allocation policies in real-time. Eventually, numerical simulations have been provided to demonstrate the effectiveness of the developed scheme.