Smart Multi-RAT Access Based on Multiagent Reinforcement Learning
Smart Multi-RAT Access Based on Multiagent Reinforcement Learning
复制标题
基于多智能体强化学习的智能多制式接入
DOI：
10.1109/tvt.2018.2793186
复制
发表时间：
2018-05-01
影响因子：
6.8
通讯作者：
Qin, Shuang
中科院分区：
文献类型：
--
作者：
Yan, Mu;Feng, Gang;Qin, Shuang
大数据时代不断增长的流量对用户体验和网络扩容提出了前所未有的需求。下一代移动网络（5g）的用户应该能够同时使用3GPP、IEEE和其他技术。将许可或非许可频段的多种无线接入技术（rat）集成已被广泛认为是大大增加网络容量的一种经济有效的方法。在本文中，我们提出了一种智能聚合RAT访问（SARA）策略，其目的是在满足不同流量服务质量（QoS）要求的同时最大化长期网络吞吐量。我们考虑了具有不同QoS需求的用户访问具有共存蜂窝wifi的异构网络的场景。为了在如此复杂和动态的环境中最大限度地提高系统吞吐量，同时满足不同的流量QoS要求，我们利用多智能体强化学习，通过感知动态信道状态和流量QoS要求，将RAT选择与单个用户访问请求的资源分配结合起来。在SARA中，我们首先使用Nash Q-learning提供一组可行的RAT选择策略，同时减少学习过程中的策略空间，然后使用基于Monte Carlo树搜索（MCTS）的Q-learning进行资源分配。数值结果表明，本文提出的SARA算法可以在有限的搜索次数下满足各种流量QoS要求的同时，最大限度地提高网络吞吐量。对于批量到达访问请求，由于实现全局最优的计算复杂度较高，可能会得到次优解。SARA的另一个吸引人的特点是，通过根据时间约束终止对MCTS的搜索，可以很容易地在解的最优性和学习时间之间进行权衡。与传统的WiFi卸载方案相比，SARA能够在保证流量QoS要求的同时显著提高网络吞吐量。
The ongoing increasing traffic in the era of big data yields unprecedented demands in user experience and network capacity expansion. The users of next generation mobile networks (5 G) should be able to use 3GPP, IEEE, and other technologies simultaneously. The integration of multiple radio access technologies (RATs) of licensed or unlicensed bands has been widely deemed as a cost-efficient way to greatly increase the network capacity. In this paper, we propose a smart aggregated RAT access (SARA) strategy with aim of maximizing the long-term network throughput while meeting diverse traffic quality of service (QoS) requirements. We consider the scenario that users with different QoS requirements access to a heterogeneous network with coexisting cellular-WiFi. In order to maximize system throughput while meeting diverse traffic QoS requirements in such a complex and dynamic environment, we exploit multiagent reinforcement learning to perform RAT selection in conjunction with resource allocation for individual user access requests, through sensing dynamic channel states and traffic QoS requirements. In SARA, we first use Nash Q-learning to provide a set of feasible RAT selection strategies while decreasing the strategy space in learning process, and then employ Monte Carlo tree search (MCTS) based Q-learning to perform resource allocation. Numerical results reveal that the network throughput can be maximized while meeting various traffic QoS requirements with limited number of searches by using our proposed SARA algorithm. For bulk arrival access requests, a suboptimal solution can be obtained as high computational complexity is incurred for achieving global optimality. Another attractive feature of SARA is that a tradeoff between the solution optimality and learning time can be readily made by terminating the search of MCTS according to the time constraint. Compared with traditional WiFi offloading schemes, SARA can significantly improve network throughput while guaranteeing traffic QoS requirements.