Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums
Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums
复制标题
DOI：
10.1137/21m1413699
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Guido Montúfar;Yue Ren;Leon Zhang
中科院分区：
文献类型：
--
作者：
Guido Montúfar;Yue Ren;Leon Zhang
我们目前的结果的线性区域的功能，可以表示人工前馈神经网络与maxout单位的数量。一个rank-k maxout单元是一个计算$k$线性函数的最大值的函数。对于具有单层maxout单元的网络，线性区域对应于多面体的Minkowski和的上顶点。我们得到的热带超曲面的相交偏序集或部分Minkowski和的上表面的数量方面的面计数公式，沿着与明确的尖锐的上限的区域的数量为任何输入尺寸，任何数量的单位，和任何职级，在有和没有偏见的情况下。基于这些结果，我们也得到了渐近尖锐的上界的网络与多层。
We present results on the number of linear regions of the functions that can be represented by artificial feedforward neural networks with maxout units. A rank-k maxout unit is a function computing the maximum of $k$ linear functions. For networks with a single layer of maxout units, the linear regions correspond to the upper vertices of a Minkowski sum of polytopes. We obtain face counting formulas in terms of the intersection posets of tropical hypersurfaces or the number of upper faces of partial Minkowski sums, along with explicit sharp upper bounds for the number of regions for any input dimension, any number of units, and any ranks, in the cases with and without biases. Based on these results we also obtain asymptotically sharp upper bounds for networks with multiple layers.