Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals
Time/Accuracy Tradeoffs for Learning a ReLU with respect to Gaussian Marginals
复制标题
DOI：
--
复制
发表时间：
2019-11
期刊：
影响因子：
--
通讯作者：
Surbhi Goel;Sushrut Karmalkar;Adam R. Klivans
中科院分区：
文献类型：
--
作者：
Surbhi Goel;Sushrut Karmalkar;Adam R. Klivans
我们认为，当根据球形高斯分布绘制示例时，在训练集上计算最合适的相对于正方形损坏的问题（标签可以是任意的）。让$ \ opt <1 $成为最合适的恢复的人口损失。我们证明：\ begin {inatizize} \ iteg finding finding s s s s s ofore-loss $ \ opt + \ epsilon $与学习稀疏的奇偶群的问题一样困难，被人们普遍认为是计算上的棘手。这是学习有关高斯边缘的恢复的第一个硬度结果，我们的结果暗示 - {\ em无条件地} - 梯度下降无法在多项式时间内收敛到全局最小值。 \ Item存在一种有效的近似算法，用于查找达到错误$ O（\ opt^{2/3}）$的最合适的relu。该算法对$ 0/1 $损失使用了新颖的半空间学习。 \ end {initizize}由于soltanolkotabi \ cite {soltanolkotabi2017llearning}的事先工作，显示梯度下降{\ em can}如果训练集是{\ em恰好}标记的，请找到相对于高斯的边缘的最佳拟合依赖。
We consider the problem of computing the best-fitting ReLU with respect to square-loss on a training set when the examples have been drawn according to a spherical Gaussian distribution (the labels can be arbitrary). Let $\opt < 1$ be the population loss of the best-fitting ReLU. We prove: \begin{itemize} \item Finding a ReLU with square-loss $\opt + \epsilon$ is as hard as the problem of learning sparse parities with noise, widely thought to be computationally intractable. This is the first hardness result for learning a ReLU with respect to Gaussian marginals, and our results imply --{\em unconditionally}-- that gradient descent cannot converge to the global minimum in polynomial time. \item There exists an efficient approximation algorithm for finding the best-fitting ReLU that achieves error $O(\opt^{2/3})$. The algorithm uses a novel reduction to noisy halfspace learning with respect to $0/1$ loss. \end{itemize} Prior work due to Soltanolkotabi \cite{soltanolkotabi2017learning} showed that gradient descent {\em can} find the best-fitting ReLU with respect to Gaussian marginals, if the training set is {\em exactly} labeled by a ReLU.