Condensing CNNs with Partial Differential Equations
Condensing CNNs with Partial Differential Equations
复制标题
DOI：
10.1109/cvpr52688.2022.00069
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Anil Kag;Venkatesh Saligrama
中科院分区：
文献类型：
--
作者：
Anil Kag;Venkatesh Saligrama
卷积神经网络（cnn）依赖于结构的深度来获取复杂的特征。它会导致低资源物联网设备的计算昂贵模型。卷积算子在接受域中是局部的，受限制的，并且随着深度的增加而增加。我们探索提供全局接受域的偏微分方程（pde），而不需要维护大型核卷积滤波器的额外开销。我们提出了一个新的特征层，称为全局层，它在特征映射上强制PDE约束，从而产生丰富的特征。通过在网络中嵌入迭代方案来解决这些约束。所提出的层可以嵌入到任何深度CNN中，将其转换为较浅的网络。因此，产生紧凑和计算效率高的体系结构，实现与原始网络相似的性能。我们的实验评估表明，具有全局层的架构所需的计算和存储预算减少了2 - 5倍，而性能没有明显损失。
Convolutional neural networks (CNNs) rely on the depth of the architecture to obtain complex features. It results in computationally expensive models for low-resource IoT devices. Convolutional operators are local and restricted in the receptive field, which increases with depth. We explore partial differential equations (PDEs) that offer a global receptive field without the added overhead of maintaining large kernel convolutional filters. We propose a new feature layer, called the Global layer, that enforces PDE constraints on the feature maps, resulting in rich features. These constraints are solved by embedding iterative schemes in the network. The proposed layer can be embedded in any deep CNN to transform it into a shallower network. Thus, resulting in compact and computationally efficient architectures achieving similar performance as the original network. Our experimental evaluation demonstrates that architectures with global layers require 2 - 5 × less computational and storage budget without any significant loss in performance.