Memory at your service: fast memory allocation for latency-critical services
Memory at your service: fast memory allocation for latency-critical services
复制标题
DOI：
10.1145/3464298.3493394
复制
发表时间：
2021-09
期刊：
影响因子：
--
通讯作者：
Aidi Pi;Junxian Zhao;Shaoqi Wang;Xiaobo Zhou
中科院分区：
文献类型：
--
作者：
Aidi Pi;Junxian Zhao;Shaoqi Wang;Xiaobo Zhou
延迟关键服务（例如键值存储和 Web 搜索）之间的共置和内存共享以及尽力而为批处理作业是提高多租户数据中心系统中内存利用率的一种有吸引力的方法。然而，我们发现，作业共置和 GNU/Linux 系统堆栈的不同目标可能会导致多租户系统中的内存压力下延迟关键型服务的性能严重下降。我们通过快速内存分配和主动回收来解决延迟关键型服务的内存压力。我们发现内存分配延迟在整体查询延迟中占主导地位，尤其是在内存压力下。我们分析了 GNU/Linux 系统堆栈提供的默认内存管理机制，并找出了它对于多租户系统中的延迟关键型服务效率低下的原因。我们提出了 Hermes，这是一种用户空间中的快速内存分配机制，可以自适应地为延迟关键型服务保留内存。它建议 Linux 操作系统主动回收批处理作业的内存。我们在 GNU C 库中实现 Hermes。实验结果表明，对于微基准测试，Hermes 将平均内存分配延迟和第 99 个百分位数的内存分配延迟分别降低了 54.4% 和 62.4%。对于两个现实世界中延迟关键的服务，Hermes 将平均尾部查询延迟和第 99 个百分点的尾部查询延迟降低了高达 40.3%。与默认的 Glibc、jemalloc 和 TCMalloc 相比，Hermes 在内存压力下将服务级别目标违规降低了高达 84.3%。
Co-location and memory sharing between latency-critical services, such as key-value store and web search, and best-effort batch jobs is an appealing approach to improving memory utilization in multi-tenant datacenter systems. However, we find that the very diverse goals of job co-location and the GNU/Linux system stack can lead to severe performance degradation of latency-critical services under memory pressure in a multi-tenant system. We address memory pressure for latency-critical services via fast memory allocation and proactive reclamation. We find that memory allocation latency dominates the overall query latency, especially under memory pressure. We analyze the default memory management mechanism provided by GNU/Linux system stack and identify the reasons why it is inefficient for latency-critical services in a multi-tenant system. We present Hermes, a fast memory allocation mechanism in user space that adaptively reserves memory for latency-critical services. It advises Linux OS to proactively reclaim memory of batch jobs. We implement Hermes in GNU C Library. Experimental result shows that Hermes reduces the average and the 99th percentile memory allocation latency by up to 54.4% and 62.4% for a micro benchmark, respectively. For two real-world latency-critical services, Hermes reduces both the average and the 99th percentile tail query latency by up to 40.3%. Compared to the default Glibc, jemalloc and TCMalloc, Hermes reduces Service Level Objective violation by up to 84.3% under memory pressure.