COLA: Orchestrating Error Coding and Learning for Robust Neural Network Inference Against Hardware Defects
COLA: Orchestrating Error Coding and Learning for Robust Neural Network Inference Against Hardware Defects
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Anlan Yu;Ning Lyu;Jieming Yin;Zhiyuan Yan;Wujie Wen
中科院分区：
文献类型：
--
作者：
Anlan Yu;Ning Lyu;Jieming Yin;Zhiyuan Yan;Wujie Wen
已经提出了纠错输出码（ECOC）来提高深度神经网络（DNN）对DNN硬件加速器的硬件缺陷的鲁棒性。不幸的是，现有的努力遭受将极大地影响其实用性的缺点：1）稳健的准确性（具有缺陷）改进，代价是降低的干净准确性（没有缺陷）; 2）不能保证使用更强的ECOC获得更好的稳健或干净准确性。在本文中，我们首先阐明了这些缺点和误差相关性之间的联系，然后提出了一种新的综合误差去相关框架，即COLA。具体来说，我们建议通过以下方式来减少内层特征误差相关性：1）采用分离的架构，其中到所有输出节点的路径的最后部分是分离的，以及2）正交化公共DNN层中的权重，使得中间特征彼此正交。我们还提出了一种基于总相关性的正则化技术，以减轻输出端的总体误差相关性。COLA的有效性首先从理论上进行分析，然后通过实验进行评估，例如，为6 .与原始DNN相比，干净的准确性提高了7%，与最先进的EECC增强DNN相比，稳健的准确性提高了40%。
Error correcting output codes (ECOCs) have been proposed to improve the robustness of deep neural networks (DNNs) against hardware defects of DNN hardware accelerators. Unfortunately, existing efforts suffer from drawbacks that would greatly impact their practicality: 1) robust accuracy (with defects) improvement at the cost of degraded clean accuracy (without defects); 2) no guarantee on better robust or clean accuracy using stronger ECOCs. In this paper, we first shed light on the connection between these drawbacks and error correlation, and then propose a novel comprehensive error decorrelation framework, namely COLA . Specifically, we propose to reduce inner layer feature error correlation by 1) adopting a separated architecture, where the last portions of the paths to all output nodes are separated, and 2) orthogonalizing weights in common DNN layers so that the intermediate features are orthogonal with each other. We also propose a regularization technique based on total correlation to mitigate overall error correlation at the outputs. The effectiveness of COLA is first analyzed theoretically, and then evaluated experimentally, e.g., up to 6 . 7% clean accuracy improvement compared with the original DNNs and up to 40% robust accuracy improvement compared to the state-of-the-art ECOC-enhanced DNNs.