Artificial Intelligence-Based Handoff Management for Dense WLANs: A Deep Reinforcement Learning Approach
Artificial Intelligence-Based Handoff Management for Dense WLANs: A Deep Reinforcement Learning Approach
复制标题
基于人工智能的密集 WLAN 切换管理：深度强化学习方法
DOI：
10.1109/access.2019.2900445
复制
发表时间：
2019-01-01
期刊：
影响因子：
3.9
通讯作者：
Guo, Lingchao
中科院分区：
文献类型：
--
作者：
Han, Zijun;Lu, Zhaoming;Guo, Lingchao
到目前为止，无线局域网(WLAN)中涉及的切换管理主要分为切换机制和判决算法。传统的切换机制在切换过程中会产生明显的延迟，导致服务不连续，这在密集的WLAN中更为明显。受软件定义网络（SDN）的启发，先前的工作提出了许多无缝切换机制来确保服务连续性。然而，对于切换决策算法，何时触发切换以及重新连接到哪个接入点仍然是棘手的问题。在本文中，我们首先设计了一种适用于基于SDN的WLAN框架的自学习架构。与此同时，我们提出了 DCRQN，一种基于深度强化学习（特别是深度 Q 网络）的新型切换管理方案。所提出的方案使网络能够从头开始学习实际用户的行为和网络状态，使其学习适应时变的密集 WLAN。由于时间相关性，切换决策被建模为马尔可夫决策过程（MDP）。在建模的 MDP 中，所提出的方案取决于决策时的实时网络统计数据。此外，利用卷积神经网络和循环神经网络来提取细粒度的判别特征。仿真数值结果表明，DCRQN能够有效提高切换过程中的数据速率，优于传统的切换方案。
So far, the handoff management involved in the wireless local area network (WLAN) has mainly fallen into the handoff mechanism and the decision algorithm. The traditional handoff mechanism generates noticeable delays during the handoff process, resulting in discontinuity of service, which is more evident in dense WLANs. Inspired by software-defined networking (SDN), prior works put forward many seamless handoff mechanisms to ensure service continuity. With respect to the handoff decision algorithm, when to trigger handoff and which access point to reconnect to, however, are still tricky problems. In this paper, we first design a self-learning architecture applicable to the SDN-based WLAN frameworks. Along with it, we propose DCRQN, a novel handoff management scheme based on deep reinforcement learning, specifically deep Q-network. The proposed scheme enables the network to learn from actual users' behaviors and network status from scratch, adapting its learning in time-varying dense WLANs. Due to the temporal correlation property, the handoff decision is modeled as the Markov decision process (MDP). In the modeled MDP, the proposed scheme depends on the real-time network statistics at the time of decisions. Moreover, the convolutional neural network and the recurrent neural network are leveraged to extract fine-grained discriminative features. The numerical results through simulation demonstrate that DCRQN can effectively improve the data rate during the handoff process, outperforming the traditional handoff scheme.