FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
复制标题
DOI：
10.1109/cvpr42600.2020.01298
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Alvin Wan;Xiaoliang Dai;Peizhao Zhang;Zijian He;Yuandong Tian;Saining Xie;Bichen Wu;Matthew Yu
中科院分区：
文献类型：
--
作者：
Alvin Wan;Xiaoliang Dai;Peizhao Zhang;Zijian He;Yuandong Tian;Saining Xie;Bichen Wu;Matthew Yu
可微神经体系结构搜索(DNAs)在设计最先进、高效的神经网络方面取得了巨大的成功。然而，与其他搜索方法相比，基于DART的DNA的搜索空间很小，因为所有候选网络层都必须在内存中显式实例化。为了解决这一瓶颈，我们提出了一种内存和计算效率高的DNA变体：DMaskingNAS。该算法将搜索空间扩展为传统DNA的10^14倍，支持在空间和通道维度上进行搜索，否则这些维度将非常昂贵：输入分辨率和过滤器数量。我们提出了一种特征映射重用的屏蔽机制，使得存储和计算代价随着搜索空间的扩展而几乎保持不变。此外，我们使用有效的形状传播来最大化每触发器或每参数的精度。与所有以前的架构相比，搜索到的FBNetV2提供了最先进的性能。DMaskingNAS查找模型的准确率比MobileNetV3-Small高0.9%，FLOPS比MobileNetV3-Small低15%，而准确率与Efficient-B0相似，但FLOPS比Efficient-B0少20%，搜索成本最高可降低421倍。此外，我们的FBNetV2在精度上比MobileNetV3高2.6%，而模型大小相当。FBNetV2型号在https://github.com/facebookresearch/mobile-vision.上是开源的
Differentiable Neural Architecture Search (DNAS) has demonstrated great success in designing state-of-the-art, efficient neural networks. However, DARTS-based DNAS's search space is small when compared to other search methods', since all candidate network layers must be explicitly instantiated in memory. To address this bottleneck, we propose a memory and computationally efficient DNAS variant: DMaskingNAS. This algorithm expands the search space by up to 10^14x over conventional DNAS, supporting searches over spatial and channel dimensions that are otherwise prohibitively expensive: input resolution and number of filters. We propose a masking mechanism for feature map reuse, so that memory and computational costs stay nearly constant as the search space expands. Furthermore, we employ effective shape propagation to maximize per-FLOP or per-parameter accuracy. The searched FBNetV2s yield state-of-the-art performance when compared with all previous architectures. With up to 421x less search cost, DMaskingNAS finds models with 0.9% higher accuracy, 15% fewer FLOPs than MobileNetV3-Small; and with similar accuracy but 20% fewer FLOPs than Efficient-B0. Furthermore, our FBNetV2 outperforms MobileNetV3 by 2.6% in accuracy, with equivalent model size. FBNetV2 models are open-sourced at https://github.com/facebookresearch/mobile-vision.