Hyperparameter optimization of deep neural network using univariate dynamic encoding algorithm for searches
Hyperparameter optimization of deep neural network using univariate dynamic encoding algorithm for searches
复制标题
DOI：
10.1016/j.knosys.2019.04.019
复制
发表时间：
2019-08
期刊：
影响因子：
--
通讯作者：
Y. Yoo
中科院分区：
文献类型：
--
作者：
Y. Yoo
本文提出了一种通过使用单变量动态编码算法进行搜索来寻找深度神经网络超参数调整的方法。优化此类神经网络的超参数很困难，因为神经网络有多个参数需要配置；此外，这种网络的训练速度很慢。该方法针对两种神经网络模型进行了测试；一个自动编码器和一个带有修改后的国家标准与技术研究所 (MNIST) 数据集的卷积神经网络。为了用所提出的方法优化超参数，自动编码器的成本函数被选择为解码值与原始图像之间的差异的平均值，以及卷积神经网络的评估精度的倒数。使用该方法对超参数进行优化，收敛速度快，计算资源少，并将结果与其他考虑的优化算法（即模拟退火、遗传算法和粒子群算法）进行比较，以显示该方法的有效性。
This paper proposes a method to find the hyperparameter tuning for a deep neural network by using a univariate dynamic encoding algorithm for searches. Optimizing hyperparameters for such a neural network is difficult because the neural network that has several parameters to configure; furthermore, the training speed for such a network is slow. The proposed method was tested for two neural network models; an autoencoder and a convolution neural network with the Modified National Institute of Standards and Technology (MNIST) dataset. To optimize hyperparameters with the proposed method, the cost functions were selected as the average of the difference between the decoded value and the original image for the autoencoder, and the inverse of the evaluation accuracy for the convolution neural network. The hyperparameters were optimized using the proposed method with fast convergence speed and few computational resources, and the results were compared with those of the other considered optimization algorithms (namely, simulated annealing, genetic algorithm, and particle swarm algorithm) to show the effectiveness of the proposed methodology.