Multilevel Artificial Neural Network Training for Spatially Correlated Learning
Multilevel Artificial Neural Network Training for Spatially Correlated Learning
复制标题
DOI：
10.1137/18m1191506
复制
发表时间：
2018-06
期刊：
影响因子：
--
通讯作者：
C. Scott;E. Mjolsness
中科院分区：
文献类型：
--
作者：
C. Scott;E. Mjolsness
多重网格建模算法是一种用于加速在类似图形结构的层次结构上运行的松弛模型的技术。我们介绍并演示了一种使用多级方法训练神经网络的新方法。使用从图距离度量导出的目标函数，我们执行正交约束优化以找到图之间的最佳延长和限制图。我们比较和对比了执行这种数值优化的几种方法，并另外提出了关于此类目标函数上限的一些新的理论结果。一旦计算出来，图之间的这些最佳映射就形成了多尺度人工神经网络（MsANN）训练的核心，这是我们提出的一种新程序，它同时训练不同空间分辨率的神经网络模型的层次结构。参数信息根据多尺度建模文献中的标准粗化和细化时间表在该层次结构的成员之间传递。在我们的机器学习实验中，这些模型能够比默认训练更快地学习，以少一个数量级的训练示例实现相当的错误水平。
Multigrid modeling algorithms are a technique used to accelerate relaxation models running on a hierarchy of similar graphlike structures. We introduce and demonstrate a new method for training neural networks which uses multilevel methods. Using an objective function derived from a graph-distance metric, we perform orthogonally-constrained optimization to find optimal prolongation and restriction maps between graphs. We compare and contrast several methods for performing this numerical optimization, and additionally present some new theoretical results on upper bounds of this type of objective function. Once calculated, these optimal maps between graphs form the core of Multiscale Artificial Neural Network (MsANN) training, a new procedure we present which simultaneously trains a hierarchy of neural network models of varying spatial resolution. Parameter information is passed between members of this hierarchy according to standard coarsening and refinement schedules from the multiscale modelling literature. In our machine learning experiments, these models are able to learn faster than default training, achieving a comparable level of error in an order of magnitude fewer training examples.