MagmaDNN: Accelerated Deep Learning Using MAGMA
MagmaDNN: Accelerated Deep Learning Using MAGMA
复制标题
DOI：
10.1145/3332186.3333047
复制
发表时间：
2019-07
期刊：
影响因子：
--
通讯作者：
Daniel Nichols;Kwai Wong;S. Tomov;Lucien Ng;Sihan Chen;Alexander Gessinger
中科院分区：
文献类型：
--
作者：
Daniel Nichols;Kwai Wong;S. Tomov;Lucien Ng;Sihan Chen;Alexander Gessinger
MagmaDNN[17]是一个深度学习框架，使用高度优化的岩浆稠密线性代数包驱动。该库提供了与其他流行框架(如TensorFlow、PyTorch和Theano)相当的性能。C++用于实现提供快速内存操作、直接CUDA访问和编译时间错误的框架。包括常见的神经网络层，如完全连通、卷积、池化、展平和丢弃。超参数调整是使用并行网格搜索引擎进行的。MagmaDNN使用几种技术来加速网络训练。例如，使用Winograd算法和FFT执行卷积。其他技术包括MagmaDNns定制内存管理器，用于减少昂贵的内存传输，以及通过跨GPU节点分发批处理来加速培训。本文概述了MagmaDNN框架及其如何利用岩浆库来实现速度提升。本文还讨论了深度网络如何通过并行训练来加速，以及并行化带来的进一步挑战。
MagmaDNN [17] is a deep learning framework driven using the highly optimized MAGMA dense linear algebra package. The library offers comparable performance to other popular frameworks, such as TensorFlow, PyTorch, and Theano. C++ is used to implement the framework providing fast memory operations, direct cuda access, and compile time errors. Common neural network layers such as Fully Connected, Convolutional, Pooling, Flatten, and Dropout are included. Hyperparameter tuning is performed with a parallel grid search engine. MagmaDNN uses several techniques to accelerate network training. For instance, convolutions are performed using the Winograd algorithm and FFTs. Other techniques include MagmaDNNs custom memory manager, which is used to reduce expensive memory transfers, and accelerated training by distributing batches across GPU nodes. This paper provides an overview of the MagmaDNN framework and how it leverages the MAGMA library to attain speed increases. This paper also addresses how deep networks are accelerated by training in parallel and further challenges with parallelization.