RepNet: Efficient On-Device Learning via Feature Reprogramming
RepNet: Efficient On-Device Learning via Feature Reprogramming
复制标题
DOI：
10.1109/cvpr52688.2022.01196
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Li Yang;A. S. Rakin;Deliang Fan
中科院分区：
文献类型：
--
作者：
Li Yang;A. S. Rakin;Deliang Fan
迁移学习的目标是将训练有素的深度学习模型从主要源任务转移到新任务，这是设备上机器学习的关键学习方案，因为物联网/边缘设备在我们的日常生活中收集并处理大量数据。然而，由于物联网/边缘设备的内存限制很小，这种设备上学习需要超小的训练内存占用，这给内存高效学习带来了新的挑战。许多现有的工作通过减少可训练参数的数量来解决这个问题。然而，这并不能直接转化为内存节省，因为主要的瓶颈是激活，而不是参数。为了开发内存高效的设备上迁移学习，在这项工作中，我们第一个从预训练模型（即主干）的中间特征重新编程的新角度来探讨迁移学习的概念。为了执行这种轻量级和内存高效的重编程，我们建议直接从新的任务输入数据中训练一个微小的重编程网络（Rep-Net），同时冻结骨干模型。提出的Rep-Net模型使用激活连接器定期与骨干模型交换特征，以使骨干模型和Rep-Net模型特征相互受益。通过广泛的实验，我们验证了所提出的Rep-Net模型的每个设计规范，以实现高内存效率的设备上重编程。我们的实验在多个基准测试中与SOTA设备上迁移学习方案相比，建立了Rep-Net的优越性能（即低训练记忆和高准确性）。代码可从https://github.com/ASU-ESIC-FAN-Lab/RepNet获得。
Transfer learning, where the goal is to transfer the well-trained deep learning models from a primary source task to a new task, is a crucial learning scheme for on-device machine learning, due to the fact that IoT/edge devices collect and then process massive data in our daily life. However, due to the tiny memory constraint in IoT/edge devices, such on-device learning requires ultra-small training memory footprint, bringing new challenges for memory-efficient learning. Many existing works solve this problem by reducing the number of trainable parameters. However, this doesn't directly translate to memory saving since the major bottleneck is the activations, not parameters. To develop memory-efficient on-device transfer learning, in this work, we are the first to approach the concept of transfer learning from a new perspective of intermediate feature re-programming of a pre-trained model (i.e., backbone). To perform this lightweight and memory-efficient reprogramming, we propose to train a tiny Reprogramming Network (Rep-Net) directly from the new task input data, while freezing the backbone model. The proposed Rep-Net model interchanges the features with the backbone model using an activation connector at regular intervals to mutually benefit both the backbone model and Rep-Net model features. Through extensive experiments, we validate each design specs of the proposed Rep-Net model in achieving highly memory-efficient on-device reprogramming. Our experiments establish the superior performance (i.e., low training memory and high accuracy) of Rep-Net compared to SOTA on-device transfer learning schemes across multiple benchmarks. Code is available at https://github.com/ASU-ESIC-FAN-Lab/RepNet.