MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning
MetaGater: Fast Learning of Conditional Channel Gated Networks via Federated Meta-Learning
复制标题
DOI：
10.1109/mass52906.2021.00031
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Sen Lin;Li Yang;Zhezhi He;Deliang Fan;Junshan Zhang
中科院分区：
文献类型：
--
作者：
Sen Lin;Li Yang;Zhezhi He;Deliang Fan;Junshan Zhang
最近，人们对用于资源受限应用的计算高效学习方法越来越感兴趣，例如，修剪、量化和信道选通。在这项工作中，我们提倡一个整体的方法来共同训练骨干网络和通道门控，可以加快子网选择的一个新的任务在资源有限的节点。特别是，我们开发了一种联合元学习算法，通过利用不同节点上的学习任务之间的模型相似性，为骨干网络和门控模块联合训练良好的元初始化。以这种方式，学习的元门控模块有效地捕获良好元骨干网络的重要滤波器，并且可以使用新任务的数据样本从元初始化快速适配任务特定的条件信道门控网络。联邦元学习算法的收敛性建立在温和的条件下。实验结果证实了我们的方法相比，相关工作的有效性。
There has recently been an increasing interest in computationally-efficient learning methods for resource-constrained applications, e.g., pruning, quantization and channel gating. In this work, we advocate a holistic approach to jointly train the backbone network and the channel gating which can speed up subnet selection for a new task at the resource-limited node. In particular, we develop a federated meta-learning algorithm to jointly train good meta-initializations for both the backbone networks and gating modules, by leveraging the model similarity across learning tasks on different nodes. In this way, the learnt meta-gating module effectively captures the important filters of a good meta-backbone network, and a task-specific conditional channel gated network can be quickly adapted from the meta-initializations using data samples of the new task. The convergence of the proposed federated meta-learning algorithm is established under mild conditions. Experimental results corroborate the effectiveness of our method in comparison to related work.