AxoNN: energy-aware execution of neural network inference on multi-accelerator heterogeneous SoCs
AxoNN: energy-aware execution of neural network inference on multi-accelerator heterogeneous SoCs
复制标题
DOI：
10.1145/3489517.3530572
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Ismet Dagli;Alexander Cieslewicz;Jedidiah McClurg;M. E. Belviranli
中科院分区：
文献类型：
--
作者：
Ismet Dagli;Alexander Cieslewicz;Jedidiah McClurg;M. E. Belviranli
嵌入式系统中关键工作负载执行（例如对象检测）的能量和延迟需求基于物理系统状态和其他外部因素而变化。许多最近的移动的和自主的片上系统（SoC）嵌入了具有独特功率和性能特性的各种加速器。关键工作负载的执行流可以调整为跨多个加速器，以便性能和能量之间的权衡适合动态变化的物理因素。在这项研究中，我们提出了运行神经网络（NN）推理的多个加速器的SoC。我们的目标是使能源性能的权衡与分布层之间的性能和功率效率的加速器NN。我们首先提供了一个经验建模方法来表征执行和层间过渡时间。然后，我们找到一个最佳的层加速器映射表示的权衡作为一个线性规划优化约束。我们评估我们的方法在NVIDIA Xavier AGX SoC与常用的NN模型。我们使用Z3 SMT求解器为不同的能耗目标找到时间表，预测准确率高达98%。
The energy and latency demands of critical workload execution, such as object detection, in embedded systems vary based on the physical system state and other external factors. Many recent mobile and autonomous System-on-Chips (SoC) embed a diverse range of accelerators with unique power and performance characteristics. The execution flow of the critical workloads can be adjusted to span into multiple accelerators so that the trade-off between performance and energy fits to the dynamically changing physical factors. In this study, we propose running neural network (NN) inference on multiple accelerators of an SoC. Our goal is to enable an energy-performance trade-off with an by distributing layers in a NN between a performance- and a power-efficient accelerator. We first provide an empirical modeling methodology to characterize execution and inter-layer transition times. We then find an optimal layers-to-accelerator mapping by representing the trade-off as a linear programming optimization constraint. We evaluate our approach on the NVIDIA Xavier AGX SoC with commonly used NN models. We use the Z3 SMT solver to find schedules for different energy consumption targets, with up to 98% prediction accuracy.