G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor Migrations
G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor Migrations
复制标题
DOI：
10.1145/3613424.3614309
复制
发表时间：
2023-10
期刊：
影响因子：
--
通讯作者：
Haoyang Zhang;Yirui Eric Zhou;Yu Xue;Yiqi Liu;Jian Huang
中科院分区：
文献类型：
--
作者：
Haoyang Zhang;Yirui Eric Zhou;Yu Xue;Yiqi Liu;Jian Huang
为了打破GPU内存墙以扩展深度学习工作负载，最近提出了各种架构和系统技术。它们的典型方法包括使用闪存和直接存储访问的存储器扩展。然而，这些技术仍然存在性能不佳的问题，并给GPU内存管理带来了复杂性，难以满足当今深度学习工作负载的可扩展性要求。在本文中，我们提出了一种名为G10的统一GPU内存和存储架构，这是由于深度学习工作负载的张量行为具有高度可预测性。G10将主机内存、GPU内存和闪存集成到一个统一的内存空间中，以扩展GPU内存容量，同时实现透明的数据迁移。基于这种统一的GPU内存和存储架构，G10利用编译器技术来表征深度学习工作负载中的张量行为。因此，它可以通过考虑闪存和主机存储器的可用带宽来提前调度数据迁移。深度学习编译器和统一内存架构之间的协作机制使G10能够以透明的方式隐藏数据传输开销。我们基于开源GPU模拟器实现了G10。我们的实验表明，G10的性能比最先进的GPU内存解决方案高出1.75倍，而无需对深度学习工作负载进行代码修改。通过智能数据迁移机制，G10可以达到理想情况下性能的90.3%（假设GPU内存无限）。CCS概念·计算机系统组织→处理器和内存架构;二级存储组织;神经网络; ·硬件→外部存储。
To break the GPU memory wall for scaling deep learning workloads, a variety of architecture and system techniques have been proposed recently. Their typical approaches include memory extension with flash memory and direct storage access. However, these techniques still suffer from suboptimal performance and introduce complexity to the GPU memory management, making them hard to meet the scalability requirement of deep learning workloads today.In this paper, we present a unified GPU memory and storage architecture named G10 driven by the fact that the tensor behaviors of deep learning workloads are highly predictable. G10 integrates the host memory, GPU memory, and flash memory into a unified memory space, to scale the GPU memory capacity while enabling transparent data migrations. Based on this unified GPU memory and storage architecture, G10 utilizes compiler techniques to characterize the tensor behaviors in deep learning workloads. Therefore, it can schedule data migrations in advance by considering the available bandwidth of flash memory and host memory. The cooperative mechanism between deep learning compilers and the unified memory architecture enables G10 to hide data transfer overheads in a transparent manner. We implement G10 based on an open-source GPU simulator. Our experiments demonstrate that G10 outperforms state-of-the-art GPU memory solutions by up to 1.75×, without code modifications to deep learning workloads. With the smart data migration mechanism, G10 can reach 90.3% of the performance of the ideal case assuming unlimited GPU memory.CCS CONCEPTS• Computer systems organization → Processors and memory architectures; Secondary storage organization; Neural networks; • Hardware → External storage.