Device Variation Effects on Neural Network Inference Accuracy in Analog In‐Memory Computing Systems
Device Variation Effects on Neural Network Inference Accuracy in Analog In‐Memory Computing Systems
复制标题
设备变化对模拟内存计算系统中神经网络推理精度的影响
DOI：
10.1002/aisy.202100199
复制
发表时间：
2022
影响因子：
7.4
通讯作者：
Lu, Wei D.
中科院分区：
文献类型：
--
作者：
Wang, Qiwen;Park, Yongmo;Lu, Wei D.
在基于非易失性存储器（例如电阻式随机存取存储器（RRAM））的模拟内存计算系统中，神经网络模型通常进行离线训练，然后将权重作为电导值编程到存储设备上。在编程过程中，编程的重量值不可避免地会偏离目标值。由于编程过程中的随机性，这种效应对于 RRAM、PcRAM 和 MRAM 等新兴存储器来说尤为明显。与噪声不同，这些权重偏差在推理过程中不会改变。我们针对现实系统限制下的编程变化来研究神经网络模型的性能，包括有限的设备开/关比、存储器阵列大小、模数转换器（ADC）特性和带符号的权重表示。还评估了通过架构感知培训来减轻此类设备和电路非理想性的方法。将讨论训练过程中变量注入对提高推理鲁棒性的有效性，以及不同神经网络训练参数（例如学习率计划）的影响。
In analog in‐memory computing systems based on nonvolatile memories such as resistive random‐access memory (RRAM), neural network models are often trained offline and then the weights are programmed onto memory devices as conductance values. The programmed weight values inevitably deviate from the target values during the programming process. This effect can be pronounced for emerging memories such as RRAM, PcRAM, and MRAM due to the stochastic nature during programming. Unlike noise, these weight deviations do not change during inference. The performance of neural network models is investigated against this programming variation under realistic system limitations, including limited device on/off ratios, memory array size, analog‐to‐digital converter (ADC) characteristics, and signed weight representations. Approaches to mitigate such device and circuit nonidealities through architecture‐aware training are also evaluated. The effectiveness of variation injection during training to improve the inference robustness, as well as the effects of different neural network training parameters such as learning rate schedule, will be discussed.
多层感知器训练期间的突触权重噪声：容错和训练改进
影响因子：
--
作者：
A. Murray;P. J. Edwards
通讯作者：
P. J. Edwards
影响因子：
34.3
作者：
Cai, Fuxi;Correll, Justin M.;Lu, Wei D.
通讯作者：
Lu, Wei D.