Many-Thread Aware Prefetching Mechanisms for GPGPU Applications
Many-Thread Aware Prefetching Mechanisms for GPGPU Applications
复制标题
DOI：
10.1109/micro.2010.44
复制
发表时间：
2010-12
期刊：
影响因子：
--
通讯作者：
Jaekyu Lee;Nagesh B. Lakshminarayana;Hyesoon Kim;R. Vuduc
中科院分区：
文献类型：
--
作者：
Jaekyu Lee;Nagesh B. Lakshminarayana;Hyesoon Kim;R. Vuduc
当应用的线程级并行性不足以隐藏内存延迟时，我们考虑了如何提高大规模多线程GPGPU的内存延迟容忍度的问题。常规CPU系统中使用的一种解决方案是硬件和软件中的预取。然而，我们表明，直接将这些机制应用于GPGPU系统并不能带来预期的性能优势，事实上，如果使用不当，可能会损害性能。本文针对GPGPU系统提出了一种新的硬件预取机制和软件预取机制，我们称之为多线程感知预取机制。我们的软件MT预取机制称为线程间预取，它利用了细粒度线程之间普遍存在的内存访问行为。对于硬件MT预取，我们描述了一种可扩展的预取器训练算法以及基于硬件的线程间预取机制。在某些情况下，盲目应用预取会降低性能。为了减少这种负面影响，我们提出了一种自适应预取节流方案，该方案允许GPGPU根据应用和硬件进行自动调整。我们的研究表明，自适应减少了预取的负面影响，甚至可以提高性能。总体而言，与最先进的软件和硬件预取相比，我们的MT预取将性能平均提高了16%(软件首选)/15%(硬件首选)。在我们的基准上。
We consider the problem of how to improve memory latency tolerance in massively multithreaded GPGPUs when the thread-level parallelism of an application is not sufficient to hide memory latency. One solution used in conventional CPU systems is prefetching, both in hardware and software. However, we show that straightforwardly applying such mechanisms to GPGPU systems does not deliver the expected performance benefits and can in fact hurt performance when not used judiciously. This paper proposes new hardware and software prefetching mechanisms tailored to GPGPU systems, which we refer to as many-thread aware prefetching (MT-prefetching) mechanisms. Our software MT-prefetching mechanism, called inter-thread prefetching, exploits the existence of common memory access behavior among fine-grained threads. For hardware MT-prefetching, we describe a scalable prefetcher training algorithm along with a hardware-based inter-thread prefetching mechanism. In some cases, blindly applying prefetching degrades performance. To reduce such negative effects, we propose an adaptive prefetch throttling scheme, which permits automatic GPGPU application- and hardware-specific adjustment. We show that adaptation reduces the negative effects of prefetching and can even improve performance. Overall, compared to the state-of-the-art software and hardware prefetching, our MT-prefetching improves performance on average by 16%(software pref.) / 15% (hardware pref.) on our benchmarks.