UVM Discard: Eliminating Redundant Memory Transfers for Accelerators
UVM Discard: Eliminating Redundant Memory Transfers for Accelerators
复制标题
DOI：
10.1109/iiswc55918.2022.00013
复制
发表时间：
2022-11
期刊：
影响因子：
--
通讯作者：
Weixi Zhu;Guilherme Cox;J. Veselý;Mark Hairgrove;A. Cox;S. Rixner
中科院分区：
文献类型：
--
作者：
Weixi Zhu;Guilherme Cox;J. Veselý;Mark Hairgrove;A. Cox;S. Rixner
越来越多的应用程序受益于异构硬件加速器。这种加速器通常需要应用程序手动管理设备上的内存缓冲区，并在主机和设备缓冲区之间传输数据。统一主机和设备之间虚拟地址空间的编程模型很有吸引力，因为它支持自动内存传输并简化了应用程序级编程。然而，自动内存传输有时可能是冗余的，这会降低性能。NVIDIA的UVM（统一虚拟内存）驱动程序为CPU-GPU编程提供了统一的虚拟地址空间。本文将冗余内存传输（RMT）确定为UVM的常见性能问题。为了解决这个问题，本文提出了一个数据丢弃指令，并评估该指令的两个实现，UvmDiscard和UvmDiscardLazy。该指令利用应用程序级知识来避免RMT。这些实现与NVIDIA的开源UVM驱动程序集成，以证明它们在现实世界的CUDA UVM应用程序中的实用性。例如，在一个过度使用GPU内存的大型深度学习应用程序上，使用discard指令可以将训练吞吐量提高61.2%。
An increasing number of applications benefit from heterogeneous hardware accelerators. Such accelerators often require the application to manually manage memory buffers on devices and transfer data between host and device buffers. A programming model that unifies the virtual address space across the host and devices is appealing because it enables automatic memory transfers and simplifies application-level programming. However, the automatic memory transfers can sometimes be redundant, which decreases performance. NVIDIA’s UVM (unified virtual memory) driver provides a unified virtual address space for CPU-GPU programming. This paper identifies redundant memory transfers (RMTs) as a common performance issue with UVM. To address this issue, this paper proposes a data discard directive, and evaluates two implementations of that directive, UvmDiscard and UvmDiscardLazy. This directive exploits application-level knowledge to avoid RMTs. The implementations were integrated with NVIDIA’s open-source UVM driver to demonstrate their usefulness on real-world CUDA UVM applications. For example, the use of the discard directive increases training throughput by 61.2% on a large deep learning application that oversubscribes GPU memory.