AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning
AutoTune: Automatically Tuning Convolutional Neural Networks for Improved Transfer Learning
复制标题
DOI：
10.1016/j.neunet.2020.10.009
复制
发表时间：
2021-01-01
期刊：
影响因子：
7.8
通讯作者：
Dubey, Shiv Ram
中科院分区：
文献类型：
--
作者：
Basha, S. H. Shabbeer;Vinakota, Sravan Kumar;Dubey, Shiv Ram
迁移学习通过使用在大规模数据集上训练的预训练深度网络来解决具有有限数据的特定任务。通常，在将学习到的知识从源任务转移到目标任务时，最后几层在目标数据集上进行微调（重新训练）。然而，这些层最初是为可能不适合目标任务的源任务设计的。在本文中，我们介绍了一种自动调整卷积神经网络（CNN）以改进迁移学习的机制。预训练的CNN层与目标数据的知识使用贝叶斯优化进行调整。首先，我们通过将softmax层中的神经元数量替换为目标任务中涉及的类别数量来训练基本CNN模型的最后一层。接下来，通过观察验证数据（贪婪标准）上的分类性能来自动调整CNN。为了评估该方法的性能，在CalTech-101、CalTech-256和Stanford Dogs三个基准数据集上进行了实验。通过本文提出的AutoTune方法获得的分类结果在三个数据集上优于标准基线迁移学习方法，分别在CalTech-101、CalTech-256和Stanford Dogs上达到95.92%、86.54%和84.67%的准确率。本研究获得的实验结果表明，使用目标数据集的知识对预训练的CNN层进行调优，具有更好的迁移学习能力。源代码可从https://github.com/JekyllAndHyde8999/AutoTune_CNN_TransferLearning获得。(c) 2020 Elsevier Ltd.版权所有。
Transfer learning enables solving a specific task having limited data by using the pre-trained deep networks trained on large-scale datasets. Typically, while transferring the learned knowledge from source task to the target task, the last few layers are fine-tuned (re-trained) over the target dataset. However, these layers are originally designed for the source task that might not be suitable for the target task. In this paper, we introduce a mechanism for automatically tuning the Convolutional Neural Networks (CNN) for improved transfer learning. The pre-trained CNN layers are tuned with the knowledge from target data using Bayesian Optimization. First, we train the final layer of the base CNN model by replacing the number of neurons in the softmax layer with the number of classes involved in the target task. Next, the CNN is tuned automatically by observing the classification performance on the validation data (greedy criteria). To evaluate the performance of the proposed method, experiments are conducted on three benchmark datasets, e.g., CalTech-101, CalTech-256, and Stanford Dogs. The classification results obtained through the proposed AutoTune method outperforms the standard baseline transfer learning methods over the three datasets by achieving 95.92%, 86.54%, and 84.67% accuracy over CalTech-101, CalTech-256, and Stanford Dogs, respectively. The experimental results obtained in this study depict that tuning of the pre-trained CNN layers with the knowledge from the target dataset confesses better transfer learning ability. The source codes are available at https://github.com/JekyllAndHyde8999/AutoTune_CNN_TransferLearning. (c) 2020 Elsevier Ltd. All rights reserved.