RL-Cache: Learning-Based Cache Admission for Content Delivery
RL-Cache: Learning-Based Cache Admission for Content Delivery
复制标题
DOI：
10.1109/jsac.2020.3000415
复制
发表时间：
2019-08
影响因子：
16.4
通讯作者：
V. Kirilin;Aditya Sundarrajan;Sergey Gorinsky;R. Sitaraman
中科院分区：
文献类型：
--
作者：
V. Kirilin;Aditya Sundarrajan;Sergey Gorinsky;R. Sitaraman
内容交付网络（cdn）通过缓存和服务用户请求的对象来分发大部分Internet内容。CDN的一个主要目标是最大化其缓存的命中率，从而使用户能够更快地下载内容。内容缓存涉及两个组件：决定是否缓存对象的接纳算法和确定在缓存已满时从缓存中取出哪个对象的驱逐算法。在本文中，我们将重点放在缓存允许上，并提出了一种称为RL- cache的新算法，该算法使用无模型强化学习（RL）来决定是否将请求的对象纳入CDN的缓存中。与之前使用少量标准进行决策的方法不同，RL-Cache对包括对象大小、近时性和访问频率在内的大量特征进行加权。我们开发了一个公开可用的RL-Cache实现，并使用Akamai CDN的图像、视频和网络流量类的生产跟踪执行评估。评估表明，与目前的技术相比，RL-Cache提高了命中率，并且只对CDN服务器施加了适度的资源开销。此外，RL-Cache足够强大，可以在一个位置进行训练，并在同一地理区域的其他位置对相同或不同流量类别的请求跟踪执行。本文还报道了RL-Cache对其特征和超参数值的敏感性的广泛分析。分析验证了所做的设计选择，并揭示了对RL-Cache行为的有趣见解。
Content delivery networks (CDNs) distribute much of the Internet content by caching and serving the objects requested by users. A major goal of a CDN is to maximize the hit rates of its caches, thereby enabling faster content downloads to the users. Content caching involves two components: an admission algorithm to decide whether to cache an object and an eviction algorithm to determine which object to evict from the cache when it is full. In this paper, we focus on cache admission and propose a novel algorithm called RL-Cache that uses model-free reinforcement learning (RL) to decide whether or not to admit a requested object into the CDN’s cache. Unlike prior approaches that use a small set of criteria for decision making, RL-Cache weights a large set of features that include the object size, recency, and frequency of access. We develop a publicly available implementation of RL-Cache and perform an evaluation using production traces for the image, video, and web traffic classes from Akamai’s CDN. The evaluation shows that RL-Cache improves the hit rate in comparison with the state of the art and imposes only a modest resource overhead on the CDN servers. Further, RL-Cache is robust enough that it can be trained in one location and executed on request traces of the same or different traffic classes in other locations of the same geographic region. The paper also reports extensive analyses of the RL-Cache sensitivity to its features and hyperparameter values. The analyses validate the made design choices and reveal interesting insights into the RL-Cache behavior.