vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design
vDNN: Virtualized deep neural networks for scalable, memory-efficient neural network design
复制标题
DOI：
10.1109/micro.2016.7783721
复制
发表时间：
2016-02
期刊：
影响因子：
--
通讯作者：
Minsoo Rhu;N. Gimelshein;Jason Clemons;A. Zulfiqar;S. Keckler
中科院分区：
文献类型：
--
作者：
Minsoo Rhu;N. Gimelshein;Jason Clemons;A. Zulfiqar;S. Keckler
使用最广泛的机器学习框架需要用户仔细调整其内存使用，以便深度神经网络（DNN）适合GPU的DRAM容量。这种限制阻碍了研究人员研究不同机器学习算法的灵活性，迫使他们要么使用不太理想的网络架构，要么在多个GPU上并行处理。我们提出了一个运行时内存管理器，虚拟化DNN的内存使用，使GPU和CPU内存可以同时用于训练更大的DNN。我们的虚拟化DNN（vDNN）将AlexNet的平均GPU内存使用量减少了89%，OverFeat减少了91%，GoogLeNet减少了95%，这大大降低了DNN的内存需求。在VGG-16上进行的类似实验，VGG-16是迄今为止最深和内存饥饿的DNN之一，证明了我们的建议的内存效率。vDNN使批量大小为256（需要28 GB内存）的VGG-16能够在包含12 GB内存的单个NVIDIA Titan X GPU卡上进行训练，与具有足够内存容纳整个DNN的假设Oracle GPU相比，性能损失18%。
The most widely used machine learning frameworks require users to carefully tune their memory usage so that the deep neural network (DNN) fits into the DRAM capacity of a GPU. This restriction hampers a researcher's flexibility to study different machine learning algorithms, forcing them to either use a less desirable network architecture or parallelize the processing across multiple GPUs. We propose a runtime memory manager that virtualizes the memory usage of DNNs such that both GPU and CPU memory can simultaneously be utilized for training larger DNNs. Our virtualized DNN (vDNN) reduces the average GPU memory usage of AlexNet by up to 89%, OverFeat by 91%, and GoogLeNet by 95%, a significant reduction in memory requirements of DNNs. Similar experiments on VGG-16, one of the deepest and memory hungry DNNs to date, demonstrate the memory-efficiency of our proposal. vDNN enables VGG-16 with batch size 256 (requiring 28 GB of memory) to be trained on a single NVIDIA Titan X GPU card containing 12 GB of memory, with 18% performance loss compared to a hypothetical, oracular GPU with enough memory to hold the entire DNN.