Noise-Resilient DNN: Tolerating Noise in PCM-Based AI Accelerators via Noise-Aware Training
Noise-Resilient DNN: Tolerating Noise in PCM-Based AI Accelerators via Noise-Aware Training
复制标题
DOI：
10.1109/ted.2021.3089987
复制
发表时间：
2021-09-01
影响因子：
3.1
通讯作者：
Burr, Geoffrey W.
中科院分区：
文献类型：
--
作者：
Kariyappa, Sanjay;Tsai, Hsinyu;Burr, Geoffrey W.
基于相变存储器（PCM）的“模拟AI”加速器对于边缘应用中的推理越来越重要，因为内存计算提供了能效。然而，PCM设备固有的噪声源会导致深度神经网络（DNN）权重值的不准确。这种不准确性可能导致模型准确性的严重退化。为了解决这个问题，我们提出了两种技术来提高DNN的噪声弹性：1）漂移正则化（DR）和2）乘性噪声训练（MNT）。我们评估了在图像分类上训练的卷积网络和在语言建模上训练的递归神经网络，并表明我们的技术在一个月内将模型准确度提高了12%。
Phase change memory (PCM)-based "Analog-AI" accelerators are gaining importance for inference in edge applications due to the energy efficiency offered by in-memory computing. Nevertheless, noise sources inherent to PCM devices cause inaccuracies in the deep neural network (DNN) weight values. Such inaccuracies can lead to severe degradation in model accuracy. To address this, we propose two techniques to improve noise resiliency of DNNs: 1) drift regularization (DR) and 2) multiplicative noise training (MNT). We evaluate convolutional networks trained on image classification and recurrent neural networks trained on language modeling and show that our techniques improve model accuracy by up to 12% over one month.