Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks.
Nonideality-Aware Training for Accurate and Robust Low-Power Memristive Neural Networks.
复制标题
DOI：
10.1002/advs.202105784
复制
发表时间：
2022-06
期刊：
影响因子：
15.1
通讯作者：
Mehonic, Adnan
中科院分区：
文献类型：
--
作者：
Joksas, Dovydas;Wang, Erwei;Barmpatsalos, Nikolaos;Ng, Wing H.;Kenyon, Anthony J.;Constantinides, George A.;Mehonic, Adnan
近年来，人工神经网络在许多认知任务中得到了迅速发展。这些结构不断增长的计算需求促成了对新技术和范例的需求，包括基于忆阻器的硬件加速器。基于忆阻交叉开关和模拟数据处理的解决方案有望提高整体能效。然而，忆阻器的非理想性会导致神经网络精度的下降，而减轻这些负面影响的尝试通常会引入设计权衡，例如功率和可靠性之间的权衡。在这项工作中，作者设计了基于忆阻器的神经网络的非理想性感知训练，能够处理最常见的设备非理想性。通过分析实验数据和采用非理想性感知训练，估计忆阻矢量矩阵乘法器的能效提高了近三个数量级（0.715 TOPs−1W−1至381 TOPs−1W−1），同时保持相似的精度。结果表明，将神经网络的参数与单个忆阻器相关联，可以通过相应的优化问题的正则化将这些设备向导电性较低的状态偏置，同时修改验证程序可以更可靠地估计性能。作者证明了这种方法的普遍性和鲁棒性时，处理范围广泛的非理想性。非理想感知训练使忆阻神经网络可行。在训练期间适应非理想性使得能够使用更节能的设备，否则由于其非线性和随机行为而难以使用。新方法是强大的，可以成功地部署，即使在不知道的确切性质的非理想性。
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever‐increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor‐based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade‐offs, such as those between power and reliability. In this work, authors design nonideality‐aware training of memristor‐based neural networks capable of dealing with the most common device nonidealities. The feasibility of using high‐resistance devices that exhibit high I‐V nonlinearity is demonstrated—by analyzing experimental data and employing nonideality‐aware training, it is estimated that the energy efficiency of memristive vector‐matrix multipliers is improved by almost three orders of magnitude (0.715 TOPs−1W−1 to 381 TOPs−1W−1) while maintaining similar accuracy. It is shown that associating the parameters of neural networks with individual memristors allows to bias these devices toward less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. The authors demonstrate the universality and robustness of this approach when dealing with a wide range of nonidealities. Nonideality‐aware training makes memristive neural networks feasible. Adapting to nonidealities during training enables to use more power‐efficient devices that would otherwise be difficult to utilize due to their nonlinear and stochastic behavior. The new method is robust and can be successfully deployed even when the exact nature of nonidealities is not known in advance.
登录
查看更多内容
影响因子：
1.3
作者：
Chang, H-Y;Narayanan, P.;Burr, G. W.
通讯作者：
Burr, G. W.
通过插入 ALD TiN 缓冲层改善基于 HfOx 的 RRAM 器件变化
影响因子：
4.9
作者：
Fang, Yichen;Yu, Zhizhen;Huang, Ru
通讯作者：
Huang, Ru
影响因子：
6.1
作者：
Ielmini, D.;Wang, Z.;Liu, Y.
通讯作者：
Liu, Y.
影响因子：
64.8
作者：
Ambrogio, Stefano;Narayanan, Pritish;Burr, Geoffrey W.
通讯作者：
Burr, Geoffrey W.
影响因子：
34.3
作者：
Cai, Fuxi;Correll, Justin M.;Lu, Wei D.
通讯作者：
Lu, Wei D.