SHF: Small: Enabling Efficient Context Switching and Effective Latency Hiding in GPUs

最初为计算机图形设计的图形处理单元（GPU）正广泛用于通用计算。该项目解决了GPU计算中的两个重要挑战。首先，它研究了使GPU能够被有效抢占的方案，这对于GPU满足云环境中的服务质量（QOS）要求至关重要。其次，该项目研究了显着提高GPU延迟隐藏能力的方法。这项跨学科的研究有两个实际用途，有效的抢占使GPU成为真正的共享资源，有效的延迟隐藏提高了GPU的性能和能源效率。研究生指导和行业合作是该项目的两个关键方面。GPU的设计理念是利用非常高的数据级并行性（DLP），表示为线程级并行性（TLP），以隐藏长指令延迟。作为副作用，GPU具有大量的片上资源来存储大量并发线程的上下文或架构状态。大的上下文导致上下文切换的延迟较长，这使得GPU难以在云服务器中真正共享。该研究项目利用单指令多线程（SIMT）执行模型的性质来大幅减少和压缩GPU上下文大小。软件和硬件的方法相结合，使防御级抢占的GPU，以满足QOS的要求。快速上下文切换还用于切换出停滞的线程并切换入新的线程，使得可以利用否则空闲的计算资源来提供高得多的延迟隐藏能力。它基本上在GPU上实现了更高的TLP，而不会扩大其关键的片上资源。

Graphics processing units (GPUs), initially designed for computer graphics, are becoming widely used for general purpose computing. This project addresses two important challenges in GPU computing. First, it investigates schemes to enable GPUs to be preempted efficiently, which is critical for GPUs to satisfy the quality of service (QOS) requirement in the cloud environment. Second, the project looks into approaches to significantly improve the latency hiding capability of GPUs. This interdisciplinary research has two practical uses, efficient preemption empowering GPUs as truly shared resource and effective latency hiding improving both the GPU performance and energy efficiency. Graduate student advising and industry collaboration are two key aspects of the project.The design philosophy of GPUs is to exploit very high degrees of data-level parallelism (DLP), expressed as thread-level parallelism (TLP), to hide long instruction latency. As a side effect, GPUs feature high amounts of on-chip resources to store the contexts or the architectural states of the large numbers of concurrent threads. The large contexts result in long latency for context switching, which makes it difficult for GPUs to be truly shared in cloud servers. This research project leverages the nature of the single-instruction multiple-thread (SIMT) execution model to drastically reduce and compress the GPU context size. Software and hardware approaches are integrated to enable instruction-level preemption for GPUs to meet the QOS requirements. Fast context switching is also used to switch out stalled threads and switch in new ones such that the otherwise idle computing resources can be utilized to provide much higher latency-hiding capability. It essentially achieves higher TLP on GPUs without enlarging their critical on-chip resources.