ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs
ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs
复制标题
DOI：
10.1109/tetc.2022.3223630
复制
发表时间：
2023-04
影响因子：
5.9
通讯作者：
B. K. Joardar;J. Doppa;Hai Helen Li;K. Chakrabarty;P. Pande
中科院分区：
文献类型：
--
作者：
B. K. Joardar;J. Doppa;Hai Helen Li;K. Chakrabarty;P. Pande
在边缘训练机器学习 (ML) 模型（最终用户设备上的片上训练）可以解决包括数据隐私/安全在内的许多紧迫挑战，通过减少对通信结构和云基础设施的依赖来增加 ML 应用程序对世界不同地区的可访问性，并满足 AR/VR 应用程序的实时要求。然而，现有的边缘平台没有足够的计算能力来支持复杂的机器学习任务，例如训练大型 CNN。基于 ReRAM 的架构为片上 CNN 训练/推理提供高性能且节能的计算平台。然而，基于 ReRAM 的架构无法随着 CNN 的大小进行扩展。更大的 CNN 具有更多的权重，这需要更多的 ReRAM 单元，而这些单元无法集成在单个芯片中。此外，在片上训练更大的 CNN 将需要更高的功率，而这些较小的设备无法承受。修剪是解决这个问题的有效方法。然而，现有的剪枝技术要么仅针对推理，要么不具有交叉感知能力。这导致在基于 ReRAM 的架构上进行 CNN 训练时，硬件节省和性能优势不是最优的。在本文中，我们通过提出一种新颖的交叉感知剪枝策略（称为 ReaLPrune）来解决这个问题，它可以剪枝超过 90% 的 CNN 权重。修剪后的模型可以从头开始训练，而不会损失任何准确性。实验结果表明，与未剪枝的 CNN 相比，ReaLPrune 降低了 77.2% 的硬件要求，并将 CNN 训练速度加快了约 20 倍。在性能和硬件节省方面，ReaLPrune 还优于其他交叉感知修剪技术。此外，ReaLPrune 对于不同的数据集和更复杂的 CNN 同样有效。
Training machine learning (ML) models at the edge (on-chip training on end user devices) can address many pressing challenges including data privacy/security, increase the accessibility of ML applications to different parts of the world by reducing the dependence on the communication fabric and the cloud infrastructure, and meet the real-time requirements of AR/VR applications. However, existing edge platforms do not have sufficient computing capabilities to support complex ML tasks such as training large CNNs. ReRAM-based architectures offer high-performance yet energy efficient computing platforms for on-chip CNN training/inferencing. However, ReRAM-based architectures are not scalable with the size of the CNN. Larger CNNs have more weights, which requires more ReRAM cells that cannot be integrated in a single chip. Moreover, training larger CNNs on-chip will require higher power, which cannot be afforded by these smaller devices. Pruning is an effective way to solve this problem. However, existing pruning techniques are either targeted for inferencing only, or they are not crossbar-aware. This leads to sub-optimal hardware savings and performance benefits for CNN training on ReRAM-based architectures. In this paper, we address this problem by proposing a novel crossbar-aware pruning strategy, referred as ReaLPrune, which can prune more than 90% of CNN weights. The pruned model can be trained from scratch without any accuracy loss. Experimental results indicate that ReaLPrune reduces hardware requirements by 77.2% and accelerates CNN training by ∼20× compared to unpruned CNNs. ReaLPrune also outperforms other crossbar-aware pruning techniques in terms of both performance and hardware savings. In addition, ReaLPrune is equally effective for diverse datasets and more complex CNNs.