Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm-猫眼课题宝

DOI：

10.48550/arxiv.2206.02976

发表时间：

2022-06

期刊：

ArXiv

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Aidan Good;Jia-Huei Lin;Hannah Sieg;Mikey Ferguson;Xin Yu;Shandian Zhe;J. Wieczorek;Thiago Serra

剪枝技术已成功应用于神经网络中，以牺牲准确性来换取稀疏性。然而，网络修剪的影响并不统一：先前的工作表明，数据集中代表性不足的类的召回可能会受到更大的负面影响。在这项工作中，我们通过假设模型固有的强化效应来研究回忆中的这种相对扭曲。也就是说，对于召回率低于准确率的类，修剪会使召回率相对较差，相反，对于召回率高于准确率的类，修剪会使召回率相对更好。此外，我们提出了一种新的剪枝算法，旨在减弱这种影响。通过统计分析，我们观察到，我们的算法的强化程度不太严重，但对于相对更困难的任务、不太复杂的模型和更高的剪枝率，强化程度更加明显。更令人惊讶的是，我们相反地观察到剪枝率较低的去强化效应，这表明适度的剪枝可能对这种扭曲具有纠正作用。

Pruning techniques have been successfully used in neural networks to trade accuracy for sparsity. However, the impact of network pruning is not uniform: prior work has shown that the recall for underrepresented classes in a dataset may be more negatively affected. In this work, we study such relative distortions in recall by hypothesizing an intensification effect that is inherent to the model. Namely, that pruning makes recall relatively worse for a class with recall below accuracy and, conversely, that it makes recall relatively better for a class with recall above accuracy. In addition, we propose a new pruning algorithm aimed at attenuating such effect. Through statistical analysis, we have observed that intensification is less severe with our algorithm but nevertheless more pronounced with relatively more difficult tasks, less complex models, and higher pruning ratios. More surprisingly, we conversely observe a de-intensification effect with lower pruning ratios, which indicates that moderate pruning may have a corrective effect to such distortions.

Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm

文献摘要

相似文献