Learning Neural Networks with Two Nonlinear Layers in Polynomial Time
Learning Neural Networks with Two Nonlinear Layers in Polynomial Time
复制标题
DOI：
--
复制
发表时间：
2017-09
期刊：
影响因子：
--
通讯作者：
Surbhi Goel;Adam R. Klivans
中科院分区：
文献类型：
--
作者：
Surbhi Goel;Adam R. Klivans
我们给出了一种多项式时算法，用于学习神经网络，其中一层sigmoids进食任何Lipschitz，单调激活函数（例如Sigmoid或Relu）。我们对网络结构没有任何假设，并且算法在$ n $ dimensions中的单位球上的{\ em any}分布成功（隐藏的重量向量也具有单位标准）。这是使用两个非线性层学习神经网络的第一个无假设，有效的算法。我们的算法 - {\ em alphatron} - 是一个简单的迭代更新规则，将等渗回归与内核方法相结合。它输出了一个假设，该假设可产生有效的Oracle访问可解释的特征。它还提出了一种新的方法，可以通过实现的条件均值功能来布尔学习问题，从而避开了计算学习理论的传统硬度。沿着这些线路，我们将PAC Learning Boolean功能的许多长期结果集成到了{\ em概率概念}的更一般，实现的设置}，该模型（与PAC学习不同）需要非I.I.I.D。噪声。
We give a polynomial-time algorithm for learning neural networks with one layer of sigmoids feeding into any Lipschitz, monotone activation function (e.g., sigmoid or ReLU). We make no assumptions on the structure of the network, and the algorithm succeeds with respect to {\em any} distribution on the unit ball in $n$ dimensions (hidden weight vectors also have unit norm). This is the first assumption-free, provably efficient algorithm for learning neural networks with two nonlinear layers. Our algorithm-- {\em Alphatron}-- is a simple, iterative update rule that combines isotonic regression with kernel methods. It outputs a hypothesis that yields efficient oracle access to interpretable features. It also suggests a new approach to Boolean learning problems via real-valued conditional-mean functions, sidestepping traditional hardness results from computational learning theory. Along these lines, we subsume and improve many longstanding results for PAC learning Boolean functions to the more general, real-valued setting of {\em probabilistic concepts}, a model that (unlike PAC learning) requires non-i.i.d. noise-tolerance.