Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization
Single-Path Mobile AutoML: Efficient ConvNet Design and NAS Hyperparameter Optimization
复制标题
DOI：
10.1109/jstsp.2020.2971421
复制
发表时间：
2019-07
影响因子：
7.5
通讯作者：
Dimitrios Stamoulis;Ruizhou Ding;Di Wang;Dimitrios Lymberopoulos;B. Priyantha;Jie Liu;Diana Marculescu
中科院分区：
文献类型：
--
作者：
Dimitrios Stamoulis;Ruizhou Ding;Di Wang;Dimitrios Lymberopoulos;B. Priyantha;Jie Liu;Diana Marculescu
我们可以将搜索成本从几天降低到几个小时，从而自动化卷积网络（Convnets）在硬件约束下的设计？ NAS问题由于组合较大的设计空间和大量的搜索时间（至少200个GPU小时）而受到挑战。在移动延迟约束下，我们提出了一种新颖的NAS公式，即单路NAS，它使用一个单路过度参数化的convnet来编码基于共享的卷积核参数的所有架构决策开销。通过将挤压和兴奋的路径视为我们新颖的单路编码的全面搜索操作，我们的方法只有8个时代（24 TPU小时），这是一个完全可搜索的操作，该操作的运行时间折衷，这是可靠的NAS，这是可靠的NAS，这是可靠的NAS，这将在可区分的NAS上进行折衷，该运行时间的总体可搜索路径（24 TPU小时），该操作是通过可识别的，这是在可区分的NAS中，该运行时间的权衡是，该运行时间折衷的NAS，该NAS的运行时间折衷，该NAS在可区分的NAS上进行了折衷。 5,000×与先前的工作相比，我们研究了不同的NAS公式选择如何影响设计的convnets的性能。 （在先前的工作中），我们可以自动找到所需的准确度折衷的高参数值（例如，对于不同平台的目标运行时）？ Cloud Automl Services，我们在以下网址开源整个代码库：https：//github.com/dstamoulis/single-path-nas。
Can we reduce the search cost of Neural Architecture Search (NAS) from days down to only a few hours? NAS methods automate the design of Convolutional Networks (ConvNets) under hardware constraints and they have emerged as key components of AutoML frameworks. However, the NAS problem remains challenging due to the combinatorially large design space and the significant search time (at least 200 GPU-hours). In this article, we alleviate the NAS search cost down to less than 3 hours, while achieving state-of-the-art image classification results under mobile latency constraints. We propose a novel differentiable NAS formulation, namely Single-Path NAS, that uses one single-path over-parameterized ConvNet to encode all architectural decisions based on shared convolutional kernel parameters, hence drastically decreasing the search overhead. Single-Path NAS achieves state-of-the-art top-1 ImageNet accuracy (75.62%), hence outperforming existing mobile NAS methods in similar latency settings ($\sim$80 ms). In particular, we enhance the accuracy-runtime trade-off in differentiable NAS by treating the Squeeze-and-Excitation path as a fully searchable operation with our novel single-path encoding. Our method has an overall cost of only 8 epochs (24 TPU-hours), which is up to 5,000× faster compared to prior work. Moreover, we study how different NAS formulation choices affect the performance of the designed ConvNets. Furthermore, we exploit the efficiency of our method to answer an interesting question: instead of empirically tuning the hyperparameters of the NAS solver (as in prior work), can we automatically find the hyperparameter values that yield the desired accuracy-runtime trade-off (e.g., target runtime for different platforms)? We view our extensive experimental results as a valuable exploration for NAS-based cloud AutoML services, and we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas.