Towards Real-Time Video Caching at Edge Servers: A Cost-Aware Deep Q-Learning Solution
Towards Real-Time Video Caching at Edge Servers: A Cost-Aware Deep Q-Learning Solution
复制标题
在边缘服务器上实现实时视频缓存：具有成本意识的深度 Q 学习解决方案
DOI：
10.1109/tmm.2021.3125803
复制
发表时间：
--
影响因子：
7.3
通讯作者：
Yuedong Xu
中科院分区：
文献类型：
--
作者：
Laizhong Cui;Erchao Ni;Yipeng Zhou;Zhi Wang;Lei Zhang;Jiangchuan Liu;Yuedong Xu
鉴于用户生成视频的快速增长，在线视频流媒体在很大程度上主导了互联网流量。在用户附近的边缘服务器上缓存视频已经成为减少主干流量和请求响应时间，提高用户端视频质量的有效方法。然而，随着时间的推移，视频的受欢迎程度可能是高度动态的。边缘服务器处的高速缓存替换的成本，特别是与替换期间的服务中断相关的成本，目前还不是很清楚。提出了一种基于边缘服务器的轻量级视频缓存算法，旨在以实时决策和最小代价优化视频缓存的命中率。受深度Q学习的启发，我们的基于DQN的在线视频缓存(DQN-OVC)有效地利用了来自用户和网络的丰富和随时可用的信息。我们将Q值函数分解为视频值函数和动作函数的乘积，大大减少了状态空间。我们实例化了低复杂度的代价感知缓存决策的操作函数，使得缓存的视频可以随着视频热度的动态变化而持续、即时地更新。我们使用了中国最大的在线视频提供商之一腾讯的视频跟踪来评估我们的DQN-OVC的性能，并将其与最先进的解决方案进行比较。结果表明，在边缘缓存环境下，DQN-OVC算法的性能明显优于基线算法。
Given the rapid growth of user-generated videos, internet traffic has been heavily dominated by online video streaming. Caching videos on edge servers in close proximity to users has been an effective approach to reduce the backbone traffic and the request response time, as well as to improve the video quality on the user side. Video popularity, however, can be highly dynamic over time. The cost of cache replacement at edge servers, particularly that related to service interruption during replacement, is not yet well understood. This paper presents a novel lightweight video caching algorithm for edge servers, seeking to optimize the hit rate with real-time decisions and minimized cost. Inspired by recent advances in deep Q-learning, our DQN-based online video caching (DQN-OVC) makes effective use of the rich and readily available information from users and networks. We decompose the Q-value function as a product of the video value function and the action function, which significantly reduces the state space. We instantiate the action function for cost-aware caching decisions with low complexity so that the cached videos can be updated continuously and instantly with dynamic video popularity. We used video traces from Tencent, one of the largest online video providers in China, to evaluate the performance of our DQN-OVC and to compare it with state-of-the-art solutions. The results demonstrate that DQN-OVC significantly outperforms the baseline algorithms in the edge caching context.