ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Model
ECC: Platform-Independent Energy-Constrained Deep Neural Network Compression via a Bilinear Regression Model
复制标题
DOI：
10.1109/cvpr.2019.01146
复制
发表时间：
2018-12
期刊：
影响因子：
--
通讯作者：
Haichuan Yang;Yuhao Zhu;Ji Liu
中科院分区：
文献类型：
--
作者：
Haichuan Yang;Yuhao Zhu;Ji Liu
许多支持DNN的视觉应用程序持续在严格的能源限制下运行，例如无人机、增强现实耳机和智能手机。设计能够满足严格能源预算的DNN正变得越来越重要。本文提出了一种压缩DNN以满足给定能量约束同时最小化精度损失的ECC框架。ECC的核心思想是通过一种新的双线性回归函数对DNN的能量消耗进行建模。能量估计模型允许我们将DNN压缩表示为在能量约束下最小化DNN损失函数的约束优化。然而，优化问题具有非常重要的约束条件。因此，现有的深度学习解算器不能直接应用。提出了一种结合乘子交替方向法(ADMM)框架和基于梯度的学习算法的优化算法。该算法将原约束优化问题分解为若干子问题，并进行迭代求解。ECC还可以跨不同的硬件平台移植，而不需要硬件知识。实验表明，与现有的资源受限的DNN压缩技术相比，ECC在相同或更低的能量消耗下获得了更高的精度。
Many DNN-enabled vision applications constantly operate under severe energy constraints such as unmanned aerial vehicles, Augmented Reality headsets, and smartphones. Designing DNNs that can meet a stringent energy budget is becoming increasingly important. This paper proposes ECC, a framework that compresses DNNs to meet a given energy constraint while minimizing accuracy loss. The key idea of ECC is to model the DNN energy consumption via a novel bilinear regression function. The energy estimate model allows us to formulate DNN compression as a constrained optimization that minimizes the DNN loss function over the energy constraint. The optimization problem, however, has nontrivial constraints. Therefore, existing deep learning solvers do not apply directly. We propose an optimization algorithm that combines the essence of the Alternating Direction Method of Multipliers (ADMM) framework with gradient-based learning algorithms. The algorithm decomposes the original constrained optimization into several subproblems that are solved iteratively and efficiently. ECC is also portable across different hardware platforms without requiring hardware knowledge. Experiments show that ECC achieves higher accuracy under the same or lower energy budget compared to state-of-the-art resource-constrained DNN compression techniques.