Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling
Dynamic Optimization of Neural Network Structures Using Probabilistic Modeling
复制标题
DOI：
10.1609/aaai.v32i1.11683
复制
发表时间：
2018-01
期刊：
影响因子：
--
通讯作者：
S. Shirakawa;Yasushi Iwata;Youhei Akimoto
中科院分区：
文献类型：
--
作者：
S. Shirakawa;Yasushi Iwata;Youhei Akimoto
深度神经网络（DNN）是强大的机器学习模型，在各种人工智能任务中取得了成功。尽管已经提出了DNN的各种架构和模块，但为目标问题选择和设计适当的网络结构是一项具有挑战性的任务。本文提出了一种在神经网络训练过程中同时优化网络结构和权值参数的方法。我们考虑一个概率分布，生成网络结构，并优化分布的参数，而不是直接优化网络结构。该方法可适用于同一框架下的各种网络结构优化问题。我们将所提出的方法应用于几个结构优化问题，如层的选择，单元类型的选择，以及使用MNIST，CIFAR-10和CIFAR-100数据集的连接选择。实验结果表明，该方法可以找到合适的和有竞争力的网络结构。
Deep neural networks (DNNs) are powerful machine learning models and have succeeded in various artificial intelligence tasks. Although various architectures and modules for the DNNs have been proposed, selecting and designing the appropriate network structure for a target problem is a challenging task. In this paper, we propose a method to simultaneously optimize the network structure and weight parameters during neural network training. We consider a probability distribution that generates network structures, and optimize the parameters of the distribution instead of directly optimizing the network structure. The proposed method can apply to the various network structure optimization problems under the same framework. We apply the proposed method to several structure optimization problems such as selection of layers, selection of unit types, and selection of connections using the MNIST, CIFAR-10, and CIFAR-100 datasets. The experimental results show that the proposed method can find the appropriate and competitive network structures.