Multi-Agent Reinforcement Learning Based Cooperative Content Caching for Mobile Edge Networks
Multi-Agent Reinforcement Learning Based Cooperative Content Caching for Mobile Edge Networks
复制标题
基于多智能体强化学习的移动边缘网络协作内容缓存
DOI：
10.1109/access.2019.2916314
复制
发表时间：
2019-01-01
期刊：
影响因子：
3.9
通讯作者：
Liu，Yijing
中科院分区：
文献类型：
--
作者：
Jiang，Wei;Feng，Gang;Liu，Yijing
为了解决以流传输视频点播文件为主的数据流量的急剧增长，可以利用移动的边缘缓存/计算（MEC）来在移动的网络边缘开发智能内容缓存，以减轻冗余流量并提高内容交付效率。在MEC架构下，内容提供商（CP）可以在MEC服务器上部署流行的视频文件，以提高用户的体验质量（QoE）。由于内容动态性、时空流量需求未知以及服务容量有限，设计高效的内容缓存策略对CP至关重要。用户偏好的知识对于高效的内容缓存非常有用和重要，但通常无法提前获得。在这种情况下，机器学习可以用于基于历史需求信息来学习用户的偏好，并决定要在MEC服务器上缓存的视频文件。在本文中，我们提出了一个多代理强化学习（MARL）为基础的合作内容缓存策略的MEC架构时，用户的偏好是未知的，只能观察到的历史内容需求。我们制定的合作内容缓存问题作为一个多智能体多臂强盗问题，并提出了一个基于MARL的算法来解决这个问题。基于MovieLens的真实的数据集进行了仿真实验，结果表明，与其他流行的内容缓存方案相比，提出的基于MARL的协同内容缓存方案可以显著降低内容下载延迟，提高内容缓存命中率.
To address the drastic growth of data traffic dominated by streaming of video-on-demand files, mobile edge caching/computing (MEC) can be exploited to develop intelligent content caching at mobile network edges to alleviate redundant traffic and improve content delivery efficiency. Under the MEC architecture, content providers (CPs) can deploy popular video files at MEC servers to improve users’ quality of experience (QoE). Designing an efficient content caching policy is crucial for CPs due to the content dynamics, unknown spatial-temporal traffic demands, and limited service capacity. The knowledge of users’ preference is very useful and important for efficient content caching, yet often unavailable in advance. Under this circumstance, machine learning can be used to learn the users’ preference based on historical demand information and decide the video files to be cached at the MEC servers. In this paper, we propose a multi-agent reinforcement learning (MARL)-based cooperative content caching policy for the MEC architecture when the users’ preference is unknown and only the historical content demands can be observed. We formulate the cooperative content caching problem as a multi-agent multi-armed bandit problem and propose a MARL-based algorithm to solve the problem. The simulation experiments are conducted based on a real dataset from MovieLens and the numerical results show that the proposed MARL-based cooperative content caching scheme can significantly reduce content downloading latency and improve content cache hit rate when compared with other popular caching schemes.