Dynamic Capacity Networks
Dynamic Capacity Networks
复制标题
DOI：
--
复制
发表时间：
2015-11
期刊：
影响因子：
--
通讯作者：
Amjad Almahairi;Nicolas Ballas;Tim Cooijmans;Yin Zheng;H. Larochelle;Aaron C. Courville
中科院分区：
文献类型：
--
作者：
Amjad Almahairi;Nicolas Ballas;Tim Cooijmans;Yin Zheng;H. Larochelle;Aaron C. Courville
我们介绍了动态容量网络（DCN），这是一种神经网络，可以在输入数据的不同部分自适应地分配其容量。这是通过组合两种类型的模块来实现的：低容量子网和高容量子网。低容量子网络应用于大部分输入，但也提供了选择输入的几个部分以应用高容量子网络的指导。选择是使用一种新的基于梯度的注意力机制，有效地识别输入区域的DCN的输出是最敏感的，我们应该投入更多的容量。我们专注于我们的经验评估的杂乱MNIST和SVHN图像数据集。我们的研究结果表明，与传统的卷积神经网络相比，DCN能够大幅减少计算次数，同时保持类似甚至更好的性能。
We introduce the Dynamic Capacity Network (DCN), a neural network that can adaptively assign its capacity across different portions of the input data. This is achieved by combining modules of two types: low-capacity sub-networks and high-capacity sub-networks. The low-capacity sub-networks are applied across most of the input, but also provide a guide to select a few portions of the input on which to apply the high-capacity sub-networks. The selection is made using a novel gradient-based attention mechanism, that efficiently identifies input regions for which the DCN's output is most sensitive and to which we should devote more capacity. We focus our empirical evaluation on the Cluttered MNIST and SVHN image datasets. Our findings indicate that DCNs are able to drastically reduce the number of computations, compared to traditional convolutional neural networks, while maintaining similar or even better performance.