MCUNet: Tiny Deep Learning on IoT Devices
MCUNet: Tiny Deep Learning on IoT Devices
复制标题
DOI：
--
复制
发表时间：
2020-07
期刊：
影响因子：
--
通讯作者：
Ji Lin;Wei-Ming Chen;Yujun Lin;J. Cohn;Chuang Gan;Song Han
中科院分区：
文献类型：
--
作者：
Ji Lin;Wei-Ming Chen;Yujun Lin;J. Cohn;Chuang Gan;Song Han
基于微控制器单元 (MCU) 的微型物联网设备上的机器学习很有吸引力，但也具有挑战性：微控制器的内存甚至比手机还要少 2-3 个数量级。我们提出了 MCUNet，这是一个联合设计高效神经架构 (TinyNAS) 和轻量级推理引擎 (TinyEngine) 的框架，可在微控制器上实现 ImageNet 规模的推理。 TinyNAS 采用两阶段神经架构搜索方法，首先优化搜索空间以适应资源限制，然后在优化的搜索空间中专门化网络架构。 TinyNAS 可以在低搜索成本下自动处理各种约束（即设备、延迟、能量、内存）。 TinyNAS 与 TinyEngine 共同设计，TinyEngine 是一个内存高效的推理库，可扩展设计空间并适应更大的模型。 TinyEngine根据整体网络拓扑而不是分层优化来调整内存调度，与TF-Lite Micro和CMSIS-NN相比，内存使用量减少了2.7倍，推理速度提高了1.7-3.3倍。 MCUNet 是第一个在现成商用微控制器上实现 >70% ImageNet top1 精度的产品，与量化的 MobileNetV2 和 ResNet-18 相比，使用的 SRAM 减少了 3.6 倍，闪存减少了 6.6 倍。在视觉唤醒词任务中，MCUNet 实现了最先进的精度，运行速度比 MobileNetV2 和基于 ProxylessNAS 的解决方案快 2.4-3.4 倍，峰值 SRAM 缩小 2.2-2.6 倍。我们的研究表明，物联网设备上始终在线的微型机器学习时代已经到来。
Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude less even than mobile phones. We propose MCUNet, a framework that jointly designs the efficient neural architecture (TinyNAS) and the lightweight inference engine (TinyEngine), enabling ImageNet-scale inference on microcontrollers. TinyNAS adopts a two-stage neural architecture search approach that first optimizes the search space to fit the resource constraints, then specializes the network architecture in the optimized search space. TinyNAS can automatically handle diverse constraints (i.e. device, latency, energy, memory) under low search costs. TinyNAS is co-designed with TinyEngine, a memory-efficient inference library to expand the design space and fit a larger model. TinyEngine adapts the memory scheduling according to the overall network topology rather than layer-wise optimization, reducing the memory usage by 2.7x, and accelerating the inference by 1.7-3.3x compared to TF-Lite Micro and CMSIS-NN. MCUNet is the first to achieves >70% ImageNet top1 accuracy on an off-the-shelf commercial microcontroller, using 3.6x less SRAM and 6.6x less Flash compared to quantized MobileNetV2 and ResNet-18. On visual wake words tasks, MCUNet achieves state-of-the-art accuracy and runs 2.4-3.4x faster than MobileNetV2 and ProxylessNAS-based solutions with 2.2-2.6x smaller peak SRAM. Our study suggests that the era of always-on tiny machine learning on IoT devices has arrived.