Selective Fine-Tuning on a Classifier Ensemble: Realizing Adaptive Neural Networks With a Diversified Multi-Exit Architecture
Selective Fine-Tuning on a Classifier Ensemble: Realizing Adaptive Neural Networks With a Diversified Multi-Exit Architecture
复制标题
DOI：
10.1109/access.2020.3047799
复制
发表时间：
2021
期刊：
影响因子：
3.9
通讯作者：
Kazutoshi Hirose;Shinya Takamaeda-Yamazaki;Jaehoon Yu;M. Motomura
中科院分区：
文献类型：
--
作者：
Kazutoshi Hirose;Shinya Takamaeda-Yamazaki;Jaehoon Yu;M. Motomura
在计算成本和推理性能之间进行权衡的自适应神经网络可能是边缘人工智能（AI）计算的关键解决方案，其中资源和能源消耗受到显着限制。边缘AI需要微调技术来实现目标精度，同时对云上的预训练模型进行更少的计算。然而，实现自适应推理成本的多出口网络需要大量的训练成本，因为它有许多需要微调的分类器。在这项研究中，我们提出了一种新的微调方法，有效地重新训练多出口网络的分类器集成。所提出的微调方法通过组装用不同的预处理数据训练的中间分类器的输出来利用个性化。评估结果表明，所提出的方法分别实现了0.2%-5.8%，0.2%-4.6%的准确率，而训练计算量仅为77%-93%，73%-84%，与预修改的CIFAR-100和Imagenet上的分类器的整体微调相比，尽管它依赖于假设的边缘环境。
Adaptive neural networks that provide a trade-off between computing costs and inference performance can be a crucial solution for edge artificial intelligence (AI) computing where resource and energy consumption are significantly constrained. Edge AIs require a fine-tuning technique to achieve target accuracy with less computation for pre-trained models on the cloud. However, a multi-exit network, which realizes adaptive inference costs, requires significant training costs because it has many classifiers that need to be fine-tuned. In this study, we propose a novel fine-tuning method for an ensemble of classifiers that efficiently retrain the multi-exit network. The proposed fine-tuning method exploits individualities by assembling the output of the intermediate classifiers trained with distinct preprocessed data. The evaluation results show that the proposed method achieved 0.2%-5.8%, 0.2%-4.6% higher accuracy with only 77%-93%, 73%-84% training computation compared to the entire fine-tuning of classifiers on the pre-modified CIFAR-100 and Imagenet, respectively, although it depends on assumed edge environments.