Towards Out-of-core Neural Networks on Microcontrollers
Towards Out-of-core Neural Networks on Microcontrollers
复制标题
DOI：
10.1109/sec54971.2022.00008
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Hongyu Miao;F. Lin
中科院分区：
文献类型：
--
作者：
Hongyu Miao;F. Lin
要在微控制器单元 (MCU) 上运行神经网络 (NN)，内存大小是主要限制。虽然算法级技术可以减少 NN 内存占用，但由此导致的 NN 准确性和通用性损失使 MCU 无法胜任许多重要用例。为了解决这一限制，我们研究了 MCU 上 NN 的核外执行：在 MCU 的小型 SRAM 与其大型低成本外部闪存之间动态交换 NN 数据块。因此，我们提出了一种调度程序设计，可以自动调度计算任务和交换 IO 任务，以最大限度地减少交换中的 IO 开销。 MCU 上的核外神经网络引起了多种担忧：执行速度减慢、存储磨损、能源消耗和数据安全。我们的实证研究表明，这些担忧都不是问题。主要好处 - MCU 能够以完全准确性/通用性运行大型神经网络 - 胜过开销。我们的研究结果表明，MCU 可以在边缘智能方面发挥更大的作用。
To run neural networks (NNs) on microcontroller units (MCUs), memory size is the major constraint. While algorithm-level techniques exist to reduce NN memory footprints, the resultant losses in NN accuracy and generality disqualify MCUs for many important use cases. To address the constraint, we investigate out-of-core execution of NNs on MCUs: dynam-ically swapping NN data tiles between an MCU's small SRAM and its large, low-cost external flash. Accordingly, we present a scheduler design that automatically schedules compute tasks and swapping IO tasks in order to minimize the IO overhead in swapping. Out-of-core NNs on MCUs raise multiple concerns: execution slowdown, storage wear out, energy consumption, and data security. Our empirical study shows that none of these concerns is a showstopper; the key benefit - MCUs being able to run large NNs with full accuracy/generality - trumps the overheads. Our findings suggest that MCUs can play a much greater role in edge intelligence.