Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms
Hyperparameter Optimization in Convolutional Neural Network using Genetic Algorithms
复制标题
使用遗传算法的卷积神经网络超参数优化
DOI：
--
复制
发表时间：
2019
影响因子：
0.9
通讯作者：
P. Dominic
中科院分区：
文献类型：
--
作者：
Nurshazlyn M. Aszemi;P. Dominic
优化卷积神经网络（CNN）中的超参数对于许多研究人员和实践者来说是一个繁琐的问题。为了获得性能更好的超参数，需要专家手动配置一组超参数选择。这种手动配置的最佳结果随后在CNN中建模和实现。然而，不同的数据集需要不同的模型或超参数组合，这可能是繁琐和乏味的。为了解决这个问题，已经提出了几个工作，如网格搜索，这是有限的低维空间，和尾巴，使用随机选择。此外，诸如进化算法和贝叶斯等优化方法已经在MNIST数据集上进行了测试，这比CIFAR-10数据集成本更低，需要的超参数更少。本文研究了CIFAR-10数据集上的超参数搜索方法。在使用各种优化方法进行调查期间，测试并记录了精度方面的性能。虽然在CIFAR-10数据集上，所提出的方法与最先进的方法之间没有显着差异，但是，实际的潜力在于遗传算法与局部搜索方法在优化网络结构和网络训练方面的混合，据作者所知，这还有待报道。
Optimizing hyperparameters in Convolutional Neural Network (CNN) is a tedious problem for many researchers and practitioners. To get hyperparameters with better performance, experts are required to configure a set of hyperparameter choices manually. The best results of this manual configuration are thereafter modeled and implemented in CNN. However, different datasets require different model or combination of hyperparameters, which can be cumbersome and tedious. To address this, several works have been proposed such as grid search which is limited to low dimensional space, and tails which use random selection. Also, optimization methods such as evolutionary algorithms and Bayesian have been tested on MNIST datasets, which is less costly and require fewer hyperparameters than CIFAR-10 datasets. In this paper, the authors investigate the hyperparameter search methods on CIFAR-10 datasets. During the investigation with various optimization methods, performances in terms of accuracy are tested and recorded. Although there is no significant difference between propose approach and the state-of-the-art on CIFAR-10 datasets, however, the actual potency lies in the hybridization of genetic algorithms with local search method in optimizing both network structures and network training which is yet to be reported to the best of author knowledge.