Enabling fast uncertainty estimation
Enabling fast uncertainty estimation
复制标题
实现快速不确定性估计
DOI：
10.1145/3489517.3530451
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Fan H
中科院分区：
文献类型：
--
作者：
Fan H
自动驾驶或医疗诊断等许多安全关键应用都需要量化神经网络 (NN) 的不确定性。最近，贝叶斯变换器展示了其提供高质量不确定性估计和出色准确性的能力。然而，它们的实时部署受到作为 Transformer 架构核心的计算密集型注意力机制以及用于量化预测不确定性的重复蒙特卡罗采样的限制。为了解决这些限制，本文通过算法和硬件优化来加速贝叶斯变换器。在算法层面，提出了一种基于进化算法（EA）的框架来利用贝叶斯变换器的稀疏性并减轻其计算工作量。在硬件层面，我们证明了稀疏性为我们优化的 CPU 和 GPU 实现带来了硬件性能的提高。还提出了一种适应性强的硬件架构来加速 FPGA 上的贝叶斯变换器。大量实验表明，基于 EA 的框架结合硬件优化，在 CPU、GPU 和 FPGA 平台上分别将贝叶斯变换器的延迟减少了多达 13 倍、12 倍和 20 倍，同时实现了更高的算法性能。
Quantifying the uncertainty of neural networks (NNs) has been required by many safety-critical applications such as autonomous driving or medical diagnosis. Recently, Bayesian transformers have demonstrated their capabilities in providing high-quality uncertainty estimates paired with excellent accuracy. However, their real-time deployment is limited by the compute-intensive attention mechanism that is core to the transformer architecture, and the repeated Monte Carlo sampling to quantify the predictive uncertainty. To address these limitations, this paper accelerates Bayesian transformers via both algorithmic and hardware optimizations. On the algorithmic level, an evolutionary algorithm (EA)-based framework is proposed to exploit the sparsity in Bayesian transformers and ease their computational workload. On the hardware level, we demonstrate that the sparsity brings hardware performance improvement on our optimized CPU and GPU implementations. An adaptable hardware architecture is also proposed to accelerate Bayesian transformers on an FPGA. Extensive experiments demonstrate that the EA-based framework, together with hardware optimizations, reduce the latency of Bayesian transformers by up to 13, 12 and 20 times on CPU, GPU and FPGA platforms respectively, while achieving higher algorithmic performance.