TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory
TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory
复制标题
DOI：
10.1145/3582016.3582063
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
H. Maruf;Hao Wang;A. Dhanotia;Johannes Weiner;Niket Agarwal;Pallab Bhattacharya;Chris Petersen;Mosharaf Chowdhury;Shobhit O. Kanaujia;Prakash Chauhan
中科院分区：
文献类型：
--
作者：
H. Maruf;Hao Wang;A. Dhanotia;Johannes Weiner;Niket Agarwal;Pallab Bhattacharya;Chris Petersen;Mosharaf Chowdhury;Shobhit O. Kanaujia;Prakash Chauhan
超大规模应用程序对内存的需求不断增加，导致内存成为整个数据中心支出的很大一部分。像CXL这样的一致接口的出现使主存扩展成为可能，并为这个问题提供了一个有效的解决方案。在这样的系统中，主存储器可以构成具有不同特性的不同存储器技术。在本文中，我们描述了内存使用模式的广泛的数据中心应用程序的服务器舰队的Meta。因此，我们展示了为这些应用程序将较冷的页面卸载到较慢的内存层的机会。然而，如果没有有效的存储器管理，这样的系统会显著降低性能。我们提出了一种新的操作系统级的应用程序透明的页面放置机制（TPP）的CXL启用的内存。TPP采用轻量级机制来识别热/冷页面并将其放置到适当的内存层。它支持从本地内存到CXL内存的主动页面降级。这种技术确保了新页面分配的内存空间，这些页面通常与请求处理相关，并且往往是短暂的和热的。与此同时，TPP可以迅速将慢速CXL-Memory中捕获的性能关键型热页提升到快速本地内存，同时最大限度地减少采样开销和不必要的迁移。TPP透明地工作，不需要任何特定于应用程序的知识，并且可以作为内核版本在全球范围内部署。我们使用支持CXL 1.1的新x86 CPU的早期样本，在生产服务器机群中评估了具有各种内存敏感型工作负载的TPP。TPP使分层内存系统的性能成为一个理想的基线（<1%的差距），所有内存都在本地层中。它比当今的Linux好18%，比包括NUMA Balancing和AutoTiering在内的现有解决方案好5-17%。大多数TPP补丁已经合并到Linux v5.18版本中，而其余的补丁还有待进一步讨论。
The increasing demand for memory in hyperscale applications has led to memory becoming a large portion of the overall datacenter spend. The emergence of coherent interfaces like CXL enables main memory expansion and offers an efficient solution to this problem. In such systems, the main memory can constitute different memory technologies with varied characteristics. In this paper, we characterize memory usage patterns of a wide range of datacenter applications across the server fleet of Meta. We, therefore, demonstrate the opportunities to offload colder pages to slower memory tiers for these applications. Without efficient memory management, however, such systems can significantly degrade performance. We propose a novel OS-level application-transparent page placement mechanism (TPP) for CXL-enabled memory. TPP employs a lightweight mechanism to identify and place hot/cold pages to appropriate memory tiers. It enables a proactive page demotion from local memory to CXL-Memory. This technique ensures a memory headroom for new page allocations that are often related to request processing and tend to be short-lived and hot. At the same time, TPP can promptly promote performance-critical hot pages trapped in the slow CXL-Memory to the fast local memory, while minimizing both sampling overhead and unnecessary migrations. TPP works transparently without any application-specific knowledge and can be deployed globally as a kernel release. We evaluate TPP with diverse memory-sensitive workloads in the production server fleet with early samples of new x86 CPUs with CXL 1.1 support. TPP makes a tiered memory system performant as an ideal baseline (<1% gap) that has all the memory in the local tier. It is 18% better than today’s Linux, and 5–17% better than existing solutions including NUMA Balancing and AutoTiering. Most of the TPP patches have been merged in the Linux v5.18 release while the remaining ones are just pending for more discussion.