Federated Deep Reinforcement Learning for THz-Beam Search with Limited CSI
Federated Deep Reinforcement Learning for THz-Beam Search with Limited CSI
复制标题
DOI：
10.1109/vtc2022-fall57202.2022.10012887
复制
发表时间：
2022-09
期刊：
影响因子：
--
通讯作者：
Po-chun Hsu;Li-Hsiang Shen;Chun-Hung Liu;Kai-Ten Feng
中科院分区：
文献类型：
--
作者：
Po-chun Hsu;Li-Hsiang Shen;Chun-Hung Liu;Kai-Ten Feng
太赫兹(THz)超宽带通信是一种很有前途的技术，可以满足下一代无线网络对高数据速率的苛刻要求，但其严重的传播衰落严重阻碍了其在实践中的实施。为大规模天线阵寻找波束方向以有效克服太赫兹信号的严重传播衰落是迫切需要的。提出了一种新的联合深度强化学习(FDRL)方法，用于快速搜索蜂窝网络中边缘服务器协调的多个基站(BS)的THz波束搜索。所有BSS都进行基于深度确定性策略梯度(DDPG)的DRL，以获得具有有限信道状态信息(CSI)的THz波束形成策略。它们使用隐藏信息更新其DDPG模型，以减轻小区间干扰。我们证明了当采用更多的太赫兹CSI和DDPG的隐含神经元时，蜂窝网络可以获得更高的吞吐量。我们还表明，部分模型更新的FDRL能够达到与完全模型更新几乎相同的性能，这表明通过部分模型上传来降低边缘服务器与BSS之间的通信负载是一种有效的手段。此外，FDRL的性能优于传统的非基于学习的基准优化方法和现有的非FDRL基准优化方法。
Terahertz (THz) communication with ultra-wide available spectrum is a promising technique that can achieve the stringent requirement of high data rate in the next-generation wireless networks, yet its severe propagation attenuation significantly hinders its implementation in practice. Finding beam directions for a large-scale antenna array to effectively overcome severe propagation attenuation of THz signals is a pressing need. This paper proposes a novel approach of federated deep reinforcement learning (FDRL) to swiftly perform THz-beam search for multiple base stations (BSs) coordinated by an edge server in a cellular network. All the BSs conduct deep deterministic policy gradient (DDPG)-based DRL to obtain THz beamforming policy with limited channel state information (CSI). They update their DDPG models with hidden information in order to mitigate inter-cell interference. We demonstrate that the cell network can achieve higher throughput as more THz CSI and hidden neurons of DDPG are adopted. We also show that FDRL with partial model update is able to nearly achieve the same performance of FDRL with full model update, which indicates an effective means to reduce communication load between the edge server and the BSs by partial model uploading. Moreover, the proposed FDRL outperforms conventional non-learning-based and existing non-FDRL benchmark optimization methods.