Fast GPU Convolution for CP-Decomposed Tensorial Neural Networks
Fast GPU Convolution for CP-Decomposed Tensorial Neural Networks
复制标题
DOI：
10.1007/978-3-030-55180-3_35
复制
发表时间：
2020-09
期刊：
影响因子：
--
通讯作者：
A. Reustle;Tahseen Rabbani;Furong Huang
中科院分区：
文献类型：
--
作者：
A. Reustle;Tahseen Rabbani;Furong Huang
提出了一种用分解张量积进行卷积的GPU算法。我们通过实验发现，对于一些张量，执行速度比Nvidia的cuDNN快4.85倍。这是通过在权张量上使用张量分解方法来扩展cnn压缩的最新进展来实现的。以前，由于缺乏快速运算来计算关键函数（如二维卷积）的分解变体，进展受到限制。我们将此操作和其他操作解释为一个复合卷积和张量收缩对分解因子(即。，广义张量运算)。之前的方法是通过组合现有库（如cuDNN）中的函数，以成对的方式评估这些网络，直到结果输出被恢复。这种评估的计算成本取决于评估指数总和的顺序，并且在不同的网络之间有所不同。两两广义张量操作的序列使计算数量最小化，通常会产生大量的中间产品，在与现代gpu的稀缺全局内存通信时产生性能瓶颈。我们的解决方案是一种GPU并行算法，该算法使用通过cp分解获得的滤波张量以最小的内存开销执行2D卷积。我们在多个分解等级下对神经网络中常见滤波器尺寸的算法的运行时性能进行基准测试。我们将自己与传统的cuDNN卷积进行比较，发现我们的实现对于较低的秩是优越的。我们还提出了一种确定成对张量操作的最佳序列的方法，在内存约束下实现最小数量的操作。
We present a GPU algorithm for performing convolution with decomposed tensor products. We experimentally find up to4.85xfaster execution times than Nvidia’s cuDNN for some tensors. This is achieved by extending recent advances in compression of CNNs through use of tensor decomposition methods on weight tensors. Progress had previously been limited by a lack of fast operations to compute the decomposed variants of critical functions such as 2D convolution. We interpret this and other operations as a network ofcompound convolution and tensor contraction on the decomposed factors(i.e.,generalized tensor operations). The prior approach sees such networks evaluated in a pairwise manner until the resulting output has been recovered, by composing functions in existing libraries such as cuDNN. The computational cost of such evaluations depends upon the order in which the index sums are evaluated, and varies between networks. The sequence of pairwise generalized tensor operations that minimizes the number of computations often produces large intermediate products, incurring performance bottlenecks when communicated with the scarce global memory of modern GPUs. Our solution is a GPU parallel algorithm which performs 2D convolution using filter tensors obtained through CP-decomposition with minimal memory overhead. We benchmark the run-time performance of our algorithm for common filter sizes in neural networks at multiple decomposition ranks. We compare ourselves against cuDNN traditional convolutions and find that our implementation is superior for lower ranks. We also propose a method for determining optimal sequences of pairwise tensor operations, achieving a minimal number of operations with memory constraints.