ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design
ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design
复制标题
DOI：
10.1109/hpca56546.2023.10071027
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Haoran You;Zhanyi Sun;Huihong Shi;Zhongzhi Yu;Yang Zhao;Yongan Zhang;Chaojian Li;Baopu Li;Yingyan Lin
中科院分区：
文献类型：
--
作者：
Haoran You;Zhanyi Sun;Huihong Shi;Zhongzhi Yu;Yang Zhao;Yongan Zhang;Chaojian Li;Baopu Li;Yingyan Lin
Vision Transformers（ViTs）在各种视觉任务上实现了最先进的性能。然而，ViTs的自我注意力模块仍然可以说是一个主要的瓶颈，限制了其可实现的硬件效率和更广泛的应用到资源受限的平台。与此同时，现有的NLP Transformer专用加速器并不适合ViT。这是因为对于自然语言处理（NLP）任务，ViT和Transformer之间存在很大的差异：ViT具有相对固定数量的输入标记，即使使用固定的稀疏模式，其注意力图也可以被修剪高达90%，而不会严重损害模型准确性（例如，=50%）。为此，我们提出了一个专用的算法和加速器协同设计框架，称为ViTCoD加速ViTs。具体而言，在算法级别上，ViTCoD修剪并极化注意力图以具有更密集或更稀疏的固定模式，用于在不损害准确性的情况下正则化两个级别的工作负载，大大减少注意力计算，同时为减轻剩余的主导数据移动留下空间;最重要的是，我们进一步集成了一个轻量级和可学习的自动编码器模块，以实现将占主导地位的高成本数据移动转换为低成本计算。在硬件层面，我们开发了一个专用加速器，以同时协调上述强制性的密集和稀疏工作负载，以提高硬件利用率，同时集成片上编码器和解码器引擎，以利用ViTCoD的算法流水线，大大减少数据移动。大量的实验和消融研究证实，ViTCoD大大降低了主要的数据移动成本，在注意力稀疏度为90%的情况下，与通用计算平台CPU、EdgeGPU、GPU以及现有技术的Transformer加速器SpAtten和桑格相比，分别实现了高达235.3×、142.9×、86.0×、10.1×和6.8×的加速。我们的代码实现可以在https://github.com/GATECH-EIC/ViTCoD上找到。
Vision Transformers (ViTs) have achieved state-of-the-art performance on various vision tasks. However, ViTs’ self-attention module is still arguably a major bottleneck, limiting their achievable hardware efficiency and more extensive applications to resource constrained platforms. Meanwhile, existing accelerators dedicated to NLP Transformers are not optimal for ViTs. This is because there is a large difference between ViTs and Transformers for natural language processing (NLP) tasks: ViTs have a relatively fixed number of input tokens, whose attention maps can be pruned by up to 90% even with fixed sparse patterns, without severely hurting the model accuracy (e.g., =50%). To this end, we propose a dedicated algorithm and accelerator co-design framework dubbed ViTCoD for accelerating ViTs. Specifically, on the algorithm level, ViTCoD prunes and polarizes the attention maps to have either denser or sparser fixed patterns for regularizing two levels of workloads without hurting the accuracy, largely reducing the attention computations while leaving room for alleviating the remaining dominant data movements; on top of that, we further integrate a lightweight and learnable auto-encoder module to enable trading the dominant high-cost data movements for lower-cost computations. On the hardware level, we develop a dedicated accelerator to simultaneously coordinate the aforementioned enforced denser and sparser workloads for boosted hardware utilization, while integrating on-chip encoder and decoder engines to leverage ViTCoD’s algorithm pipeline for much reduced data movements. Extensive experiments and ablation studies validate that ViTCoD largely reduces the dominant data movement costs, achieving speedups of up to 235.3×, 142.9×, 86.0×, 10.1×, and 6.8× over general computing platforms CPUs, EdgeGPUs, GPUs, and prior-art Transformer accelerators SpAtten and Sanger under an attention sparsity of 90%, respectively. Our code implementation is available at https://github.com/GATECH-EIC/ViTCoD.