Analyzing and Leveraging Shared L1 Caches in GPUs
Analyzing and Leveraging Shared L1 Caches in GPUs
复制标题
分析和利用 GPU 中的共享 L1 缓存
DOI：
10.1145/3410463.3414623
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Jog, Adwait
中科院分区：
文献类型：
--
作者：
Ibrahim, Mohamed Assem;Kayiran, Onur;Eckert, Yasuko;Loh, Gabriel H.;Jog, Adwait
图形处理单元（GPU）同时执行数千个线程，这使得它们能够有效地为各种应用程序实现高吞吐量。然而，内存墙通常限制峰值吞吐量。GPU使用缓存来解决这个限制，因此之前的几项工作都集中在提高缓存命中率上，这反过来又可以提高内存密集型应用程序的吞吐量。然而，几乎所有的现有工作都假设传统的高速缓存层次结构，其中每个GPU核心具有私有本地L1高速缓存，并且所有核心共享L2高速缓存。我们的分析表明，这种规范的组织不允许缓存的最佳利用率，因为L1缓存的私有性质允许多个副本的同一个缓存线得到复制跨core.We引入一个新的共享L1缓存组织，所有的核心集体缓存的数据的一个副本在只有一个位置（核心），导致零数据复制。我们通过允许每个核心只缓存整个地址范围的非重叠切片来实现这一点。这样的设计对于显著提高集体L1命中率是有用的，但是当核请求不允许存在于其自己的高速缓存中的数据时，会招致来自附加通信的延迟开销。虽然许多工作负载可以容忍这种额外的延迟，但一些工作负载表现出性能敏感性。因此，我们开发轻量级的通信优化技术和运行时的机制，认为延迟容忍特性的应用程序，以决定哪些应用程序应该执行私人与共享L1缓存组织和重新配置相应的缓存。实际上，对于喜欢共享组织的应用程序，我们以适度的硬件成本实现了显着的性能和能效改进，对其他应用程序几乎没有影响。
Graphics Processing Units (GPUs) concurrently execute thousands of threads, which makes them effective for achieving high throughput for a wide range of applications. However, the memory wall often limits peak throughput. GPUs use caches to address this limitation, and hence several prior works have focused on improving cache hit rates, which in turn can improve throughput for memory-intensive applications. However, almost all of the prior works assume a conventional cache hierarchy where each GPU core has a private local L1 cache and all cores share the L2 cache. Our analysis shows that this canonical organization does not allow optimal utilization of caches because the private nature of L1 caches allows multiple copies of the same cache line to get replicated across cores.We introduce a new shared L1 cache organization, where all cores collectively cache a single copy of the data at only one location (core), leading to zero data replication. We achieve this by allowing each core to cache only a non-overlapping slice of the entire address range. Such a design is useful for significantly improving the collective L1 hit rates but incurs latency overheads from additional communications when a core requests data not allowed to be present in its own cache. While many workloads can tolerate this additional latency, several workloads show performance sensitivities. Therefore, we develop lightweight communication optimization techniques and a run-time mechanism that considers the latency-tolerance characteristics of applications to decide which applications should execute in private versus shared L1 cache organization and reconfigures the caches accordingly. In effect, we achieve significant performance and energy efficiency improvements, at a modest hardware cost, for applications that prefer the shared organization, with little to no impact on other applications.
登录
查看更多内容
均衡器：动态调整 GPU 资源以实现高效执行
DOI：
10.1109/micro.2014.16
发表时间：
2014
期刊：
2014 47th Annual IEEE/ACM International Symposium on Microarchitecture
影响因子：
--
作者：
Ankit Sethia;S. Mahlke
通讯作者：
S. Mahlke
Pomegranate：完全可扩展的图形架构
DOI：
10.1145/344779.344981
发表时间：
2000
期刊：
Proceedings of the 27th annual conference on Computer graphics and interactive techniques
影响因子：
--
作者：
Matthew Eldridge;Homan Igehy;P. Hanrahan
通讯作者：
P. Hanrahan
DOI：
10.1109/hpca.2018.00030
发表时间：
2018-02
期刊：
2018 IEEE International Symposium on High Performance Computer Architecture (HPCA)
影响因子：
--
作者：
Haonan Wang;Fan Luo;M. Ibrahim;Onur Kayiran;Adwait Jog
通讯作者：
Haonan Wang;Fan Luo;M. Ibrahim;Onur Kayiran;Adwait Jog
DOI：
10.1109/micro.2014.62
发表时间：
2014-12
期刊：
2014 47th Annual IEEE/ACM International Symposium on Microarchitecture
影响因子：
--
作者：
Onur Kayiran;N. Nachiappan;Adwait Jog;Rachata Ausavarungnirun;M. Kandemir;G. Loh;O. Mutlu;C. Das
通讯作者：
Onur Kayiran;N. Nachiappan;Adwait Jog;Rachata Ausavarungnirun;M. Kandemir;G. Loh;O. Mutlu;C. Das
Nexus：分布式共享缓存中复制的新方法
DOI：
10.1109/pact.2017.42
发表时间：
2017
期刊：
2017 26th International Conference on Parallel Architectures and Compilation Techniques (PACT)
影响因子：
--
作者：
Po;Nathan Beckmann;Daniel Sánchez
通讯作者：
Daniel Sánchez