Joint Relay Selection and Beam Management Based on Deep Reinforcement Learning for Millimeter Wave Vehicular Communication
Joint Relay Selection and Beam Management Based on Deep Reinforcement Learning for Millimeter Wave Vehicular Communication
复制标题
DOI：
10.1109/tvt.2023.3274763
复制
发表时间：
2022-12
影响因子：
6.8
通讯作者：
Dohyun Kim;Miguel R. Castellanos;R. Heath
中科院分区：
文献类型：
--
作者：
Dohyun Kim;Miguel R. Castellanos;R. Heath
协作中继通过提供用于数据传输的多个路径来提高无线网络中的可靠性和覆盖。中继将在更高频段的车载网络中发挥重要作用，其中移动性和频繁的信号阻塞会导致链路中断。为了确保中继辅助车载网络的连通性，中继选择策略应被设计为有效地找到未阻塞的中继。受移动的毫米波（mmWave）网络中波束管理的最新进展的启发，本文解决了以下问题：如何在波束管理的开销最小的情况下选择最佳中继？在这方面，我们制定了一个顺序决策问题，以共同优化中继选择和波束管理。我们提出了一种基于深度强化学习（DRL）的联合中继选择和波束管理策略，使用波束索引和波束测量的马尔可夫特性。建议的DRL为基础的算法学习适应动态信道条件和业务模式的时变阈值。数值实验表明，该算法优于基线没有事先信道知识。此外，基于DRL的算法可以在快速变化的信道下保持高的频谱效率。
Cooperative relays improve reliability and coverage in wireless networks by providing multiple paths for data transmission. Relaying will play an essential role in vehicular networks at higher frequency bands, where mobility and frequent signal blockages cause link outages. To ensure connectivity in a relay-aided vehicular network, the relay selection policy should be designed to efficiently find unblocked relays. Inspired by recent advances in beam management in mobile millimeter wave (mmWave) networks, this article addresses the question: how can the best relay be selected with minimal overhead from beam management? In this regard, we formulate a sequential decision problem to jointly optimize relay selection and beam management. We propose a joint relay selection and beam management policy based on deep reinforcement learning (DRL) using the Markov property of beam indices and beam measurements. The proposed DRL-based algorithm learns time-varying thresholds that adapt to the dynamic channel conditions and traffic patterns. Numerical experiments demonstrate that the proposed algorithm outperforms baselines without prior channel knowledge. Moreover, the DRL-based algorithm can maintain high spectral efficiency under fast-varying channels.