Learning across scales - A multiscale method for Convolution Neural Networks
Learning across scales - A multiscale method for Convolution Neural Networks
复制标题
跨尺度学习 - 卷积神经网络的多尺度方法
DOI：
--
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
E. Holtham
中科院分区：
文献类型：
--
作者：
E. Haber;Lars Ruthotto;E. Holtham
在这项工作中，我们建立了最优控制和训练深度卷积神经网络（CNN）之间的关系。我们证明了CNN中的前向传播可以被解释为一个依赖于时间的非线性微分方程，学习是控制微分方程的参数，使得网络近似于给定训练数据的数据标签关系。使用这种连续的解释，我们得到了两种新的方法来缩放CNN相对于两个不同的维度。第一类多尺度方法通过CNN参数的延长和限制来连接低分辨率和高分辨率数据。我们证明，这使得能够使用低分辨率图像训练的CNN对高分辨率图像进行分类，反之亦然，并热启动学习过程。第二类多尺度方法连接浅层和深层网络，并导致新的训练策略，逐渐增加CNN的深度，同时重新使用参数进行初始化。
In this work we establish the relation between optimal control and training deep Convolution Neural Networks (CNNs). We show that the forward propagation in CNNs can be interpreted as a time-dependent nonlinear differential equation and learning as controlling the parameters of the differential equation such that the network approximates the data-label relation for given training data. Using this continuous interpretation we derive two new methods to scale CNNs with respect to two different dimensions. The first class of multiscale methods connects low-resolution and high-resolution data through prolongation and restriction of CNN parameters. We demonstrate that this enables classifying high-resolution images using CNNs trained with low-resolution images and vice versa and warm-starting the learning process. The second class of multiscale methods connects shallow and deep networks and leads to new training strategies that gradually increase the depths of the CNN while re-using parameters for initializations.
影响因子：
3.3
作者：
Warner, Michael;Ratcliffe, Andrew;Bertrand, Alexandre
通讯作者：
Bertrand, Alexandre