Parameter Density Inheritance Using Kernel Density Estimation for Efficient CNN Learning
Parameter Density Inheritance Using Kernel Density Estimation for Efficient CNN Learning
复制标题
使用核密度估计的参数密度继承实现高效 CNN 学习
DOI：
10.1109/isspit.2018.8642618
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Keisuke Horiuchi and Keisuke Kameyama
中科院分区：
文献类型：
--
作者：
越中彩貴;矢内浩文;Keisuke Horiuchi and Keisuke Kameyama
CNN 在多种分类任务上都表现出了最先进的性能。然而，由于其复杂性，它们可能需要大量时间进行训练并且仍然无法收敛。解决这些问题的关键思想之一是在训练之前用适当的值初始化网络参数。几种已知的方法提出使用现有的训练网络进行初始化，但它们假设训练网络和新网络具有相同的结构。在本文中，我们提出了一种利用现有训练网络的初始化方法，并且可以基于参数密度继承策略应用于具有不同结构的网络。在实验中，我们验证了所提出的初始化方法的有效性。当训练后的网络和新网络的网络结构相同时，结果表明现有方法之一优于所提出的方法。然而，在网络结构不相同的情况下，使用所提出的方法观察到了改进。
CNNs have shown state-of-the-art performances on a large variety of classification tasks. However, because of its complexity, they may require an enormous amount of time for training and still not converge. One of the key ideas to address these problems is to initialize the network parameters with appropriate values before training. Several known methods proposed the initialization using an existing trained network, but they suppose both the trained network and the new network have the same structure. In this paper, we propose an initialization method that utilizes existing trained network and can be applied to a network with a different structure based on parameter density inheritance strategy. In the experiments, we verified the effectiveness of the proposed initialization method. When the network structures were the same between the trained network and the new network, the result showed that one of the existing methods is better than the proposed method. However, in the case that network structures are not the same, an improvement was observed using the proposed method.