On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networks
On transversality of bent hyperplane arrangements and the topological expressiveness of ReLU neural networks
复制标题
DOI：
10.1137/20m1368902
复制
发表时间：
2020-08
期刊：
影响因子：
--
通讯作者：
J. E. Grigsby;Kathryn A. Lindsey
中科院分区：
文献类型：
--
作者：
J. E. Grigsby;Kathryn A. Lindsey
设F：R^n -> R是一个前馈ReLU神经网络。众所周知，对于任何参数的选择，F是连续的和分段（仿射）线性的。我们奠定了一些基础，系统的调查F的架构如何影响其可能的二进制分类任务的决策区域的几何形状和拓扑结构。遵循微分拓扑中光滑函数的经典进展，我们首先定义了通用的、横向的ReLU神经网络的概念，并证明了几乎所有的ReLU网络都是通用的和横向的。然后，我们定义了一个部分定向的线性1-复域的F和确定的属性，这个复杂的决策区域的有界连通分量的存在产生的障碍。我们使用这个障碍来证明一个通用的，横向的ReLU网络F：R^n -> R的决策区域具有一个维度为（n + 1）的单个隐藏层，可以有不超过一个有界连通分量。
Let F:R^n -> R be a feedforward ReLU neural network. It is well-known that for any choice of parameters, F is continuous and piecewise (affine) linear. We lay some foundations for a systematic investigation of how the architecture of F impacts the geometry and topology of its possible decision regions for binary classification tasks. Following the classical progression for smooth functions in differential topology, we first define the notion of a generic, transversal ReLU neural network and show that almost all ReLU networks are generic and transversal. We then define a partially-oriented linear 1-complex in the domain of F and identify properties of this complex that yield an obstruction to the existence of bounded connected components of a decision region. We use this obstruction to prove that a decision region of a generic, transversal ReLU network F: R^n -> R with a single hidden layer of dimension (n + 1) can have no more than one bounded connected component.