Attentive Normalization
Attentive Normalization
复制标题
DOI：
10.1007/978-3-030-58520-4_5
复制
发表时间：
2019-08
期刊：
影响因子：
--
通讯作者：
Xilai Li;Wei Sun-;Tianfu Wu
中科院分区：
文献类型：
--
作者：
Xilai Li;Wei Sun-;Tianfu Wu
在最先进的深度神经网络中，特征归一化和特征关注已经变得无处不在。然而，它们通常作为单独的模块进行研究。在本文中，我们提出了两个模式之间的轻量级集成和当前的注意归一化（AN）。AN不是学习单一的仿射变换，而是学习混合的仿射变换，并利用它们的加权和作为最终的仿射变换，以特定于实例的方式重新校准特征。权重是通过利用渠道特征注意力来学习的。在实验中，我们在ImageNet-1000分类基准和MS-COCO 2017目标检测和实例分割基准中使用四种具有代表性的神经结构来测试所提出的AN。在两个基准测试中，AN在不同的神经架构上获得了一致的性能提升，ImageNet-1000的top-1精度绝对提升在0.5%到2.7%之间，MS-COCO的bounding box和mask AP的绝对提升分别高达1.8%和2.2%。我们观察到，所提出的AN为广泛使用的挤压和激励（SE）模块提供了一个强大的替代方案。源代码可在ImageNet分类报告和MS-COCO检测和分割报告中公开获得。
In state-of-the-art deep neural networks, both feature normalization and feature attention have become ubiquitous. They are usually studied as separate modules, however. In this paper, we propose a light-weight integration between the two schema and present Attentive Normalization (AN). Instead of learning a single affine transformation, AN learns a mixture of affine transformations and utilizes their weighted-sum as the final affine transformation applied to re-calibrate features in an instance-specific way. The weights are learned by leveraging channel-wise feature attention. In experiments, we test the proposed AN using four representative neural architectures in the ImageNet-1000 classification benchmark and the MS-COCO 2017 object detection and instance segmentation benchmark. AN obtains consistent performance improvement for different neural architectures in both benchmarks with absolute increase of top-1 accuracy in ImageNet-1000 between 0.5% and 2.7%, and absolute increase up to 1.8% and 2.2% for bounding box and mask AP in MS-COCO respectively. We observe that the proposed AN provides a strong alternative to the widely used Squeeze-and-Excitation (SE) module. The source codes are publicly available at the ImageNet Classification Repo and the MS-COCO Detection and Segmentation Repo .