Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks
Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks
复制标题
DOI：
10.4208/cicp.oa-2020-0214
复制
发表时间：
2018-05
期刊：
影响因子：
--
通讯作者：
Yingzhou Li;Xiuyuan Cheng;Jianfeng Lu
中科院分区：
文献类型：
--
作者：
Yingzhou Li;Xiuyuan Cheng;Jianfeng Lu
深度网络，特别是卷积神经网络（CNN），已成功应用于机器学习的各个领域以及其他科学和工程领域的挑战性问题。本文介绍了Butterfly-net，这是一种具有结构化和稀疏跨通道连接的低复杂度CNN，其目标是输入信号的最佳分层函数表示。理论分析表明，蝶形网对输入数据的傅立叶表示的逼近能力随深度的增加呈指数衰减。由于蝶形网近似傅立叶和局部傅立叶变换的能力，该结果可以用于一大类问题中CNN的近似上限。通过对一维傅立叶核和一维、二维泊松方程能量近似的数值实验，验证了分析结果的正确性。具有训练参数的蝶形网优于硬编码的蝶形网，并达到与训练CNN相似的精度，但参数要少得多。此外，当输入数据的分布具有域移位时，蝶形网对CNN具有更好的鲁棒性。
Deep networks, especially Convolutional Neural Networks (CNNs), have been successfully applied in various areas of machine learning as well as to challenging problems in other scientific and engineering fields. This paper introduces Butterfly-net, a low-complexity CNN with structured and sparse across-channel connections, which aims at an optimal hierarchical function representation of the input signal. Theoretical analysis of the approximation power of Butterfly-net to the Fourier representation of input data shows that the error decays exponentially as the depth increases. Due to the ability of Butterfly-net to approximate Fourier and local Fourier transforms, the result can be used for approximation upper bound for CNNs in a large class of problems. The analytical results are validated by numerical experiments on the approximation of a 1D Fourier kernel and of the energy of 1D and 2D Poisson's equations. Butterfly-net with trained parameters outperforms the hard-coded Butterfly-net and achieves similar accuracy as the trained CNN but with much less parameters. In addition, better robustness of Butterfly-net against CNN is demonstrated when the distribution of the input data has domain shift.