Neural Networks with Local Receptive Fields and Superlinear VC Dimension
Neural Networks with Local Receptive Fields and Superlinear VC Dimension
复制标题
DOI：
10.1162/089976602317319018
复制
发表时间：
2002-04
影响因子：
2.9
通讯作者：
M. Schmitt
中科院分区：
文献类型：
--
作者：
M. Schmitt
局部感受区神经元包括径向基函数(RBF)神经元和中心环绕感受区神经元等广泛使用的单位类型。研究了具有一个隐层的前馈神经网络的Vapnik-Chervonenkis(VC)维。对于几种不同的局部感受场神经元，我们证明了这些网络的VC维是超线性的。特别地，我们建立了任意具有W个参数和k个隐节点的合理规模的网络的界(Wlogk)。这一界限被证明适用于离散的中心-周围感受野神经元，它是哺乳动物视觉系统中细胞的生理相关模型，计算高斯差的神经元，在计算视觉中很受欢迎，以及标准的RBF神经元，它是人工神经网络中S型神经元的主要替代品。RBF神经网络的结果特别令人感兴趣，因为它回答了一个多年来一直悬而未决的问题。这些结果还给出了固定输入维度网络的下界。关于常数，所有的界限都比迄今为止已知的具有乙状结节神经元的类似结构的界限大。单个局部感受区神经元的超线性下界和线性上界也在这里得到了对比。
Local receptive field neurons comprise such well-known and widely used unit types as radial basis function (RBF) neurons and neurons with center-surround receptive field. We study the Vapnik-Chervonenkis (VC) dimension of feedforward neural networks with one hidden layer of these units. For several variants of local receptive field neurons, we show that the VC dimension of these networks is superlinear. In particular, we establish the bound (w log k) for any reasonably sized network with W parameters and k hidden nodes. This bound is shown to hold for discrete center-surround receptive field neurons, which are physiologically relevant models of cells in the mammalian visual system, for neurons computing a difference of gaussians, which are popular in computational vision, and for standard RBF neurons, a major alternative to sigmoidal neurons in artificial neural networks. The result for RBF neural networks is of particular interest since it answers a question that has been open for several years. The results also give rise to lower bounds for networks with fixed input dimension. Regarding constants, all bounds are larger than those known thus far for similar architectures with sigmoidal neurons. The superlinear lower bounds contrast with linear upper bounds for single local receptive field neurons also derived here.