GPUReplay: a 50-KB GPU stack for client ML
GPUReplay: a 50-KB GPU stack for client ML
复制标题
DOI：
10.1145/3503222.3507754
复制
发表时间：
2021-05
期刊：
影响因子：
--
通讯作者：
Heejin Park;F. Lin
中科院分区：
文献类型：
--
作者：
Heejin Park;F. Lin
GPUReplay （GR）是一种在移动和嵌入式设备上部署gpu加速计算的新方法。它解决了现代GPU堆栈的高度复杂性，以实现部署的便利性和安全性。这个想法是提前在完整的GPU堆栈上记录GPU执行，并在运行时重播新输入的执行。我们解决了使GR可行、合理和实用的关键挑战。由此产生的重放器是原始GPU堆栈的替换。它很小（50 KB的可执行文件）、健壮（重放长时间的执行而不产生差异）、可移植（在普通操作系统、TEE和裸机中运行）、启动速度快（将启动速度提高两个数量级）。我们展示了GPUReplay可以与各种集成的GPU硬件、GPU api、ML框架和33个神经网络（NN）实现一起工作，用于推理或训练。代码可在https://github.com/bakhi/GPUReplay上获得。
GPUReplay (GR) is a novel way for deploying GPU-accelerated computation on mobile and embedded devices. It addresses high complexity of a modern GPU stack for deployment ease and security. The idea is to record GPU executions on the full GPU stack ahead of time and replay the executions on new input at run time. We address key challenges towards making GR feasible, sound, and practical to use. The resultant replayer is a drop-in replacement of the original GPU stack. It is tiny (50 KB of executable), robust (replaying long executions without divergence), portable (running in a commodity OS, in TEE, and baremetal), and quick to launch (speeding up startup by up to two orders of magnitude). We show that GPUReplay works with a variety of integrated GPU hardware, GPU APIs, ML frameworks, and 33 neural network (NN) implementations for inference or training. The code is available at https://github.com/bakhi/GPUReplay.