Poise: Balancing Thread-Level Parallelism and Memory System Performance in GPUs Using Machine Learning
Poise: Balancing Thread-Level Parallelism and Memory System Performance in GPUs Using Machine Learning
复制标题
DOI：
10.1109/hpca.2019.00061
复制
发表时间：
2019-02
期刊：
影响因子：
--
通讯作者：
Saumay Dublish;V. Nagarajan;N. Topham
中科院分区：
文献类型：
--
作者：
Saumay Dublish;V. Nagarajan;N. Topham
- GPUS员工高度的线程级并联（TLP）隐藏了记忆操作的长期延迟，TLP的高度可能会对系统的吞吐量产生不利影响。组件：机器学习框架和硬件推理引擎。因此，动态的预先扭曲计划对看不见的应用程序有助于优化全新的应用程序训练或编程在最终用户上燃烧。在训练期间看不见的一组基准，在训练期间，训练的速度最高为2.94倍，而谐波平均速度为46.6％，超过了基线贪婪 - 最古老的扭曲调度程序。平衡非常轻巧，每SM的最小硬件开销也使整体能源消耗量平均降低了51.6％。最先进的扭曲调度程序平均有15.1％，以解决一个复杂的硬件优化问题，并具有能力的准确性和效率。
—GPUs employ a high degree of thread-level paral- lelism (TLP) to hide the long latency of memory operations. However, the consequent increase in demand on the memory system causes pathological effects such as cache thrashing and bandwidth bottlenecks. As a result, high degrees of TLP can adversely affect system throughput. In this paper, we present Poise , a novel approach for balancing TLP and memory system performance in GPUs. Poise has two major components: a machine learning framework and a hardware inference engine. The machine learning framework comprises a regression model that is trained offline on a set of profiled kernels to learn best warp scheduling decisions. At runtime, the hardware inference engine uses the previously learned model to dynamically pre- dict best warp scheduling decisions for unseen applications. Therefore, Poise helps in optimizing entirely new applications without posing any profiling, training or programming burden on the end-user. Across a set of benchmarks that were unseen during training, Poise achieves a speedup of up to 2.94 × and a harmonic mean speedup of 46.6%, over the baseline greedy- then-oldest warp scheduler. Poise is extremely lightweight and incurs a minimal hardware overhead of around 41 bytes per SM. It also reduces the overall energy consumption by an average of 51.6%. Furthermore, Poise outperforms the prior state-of- the-art warp scheduler by an average of 15.1%. In effect, Poise solves a complex hardware optimization problem with consider- able accuracy and efficiency.