Adversarial Examples in Multi-Layer Random ReLU Networks
Adversarial Examples in Multi-Layer Random ReLU Networks
复制标题
DOI：
--
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
P. Bartlett;Sébastien Bubeck;Yeshwanth Cherapanamjeri
中科院分区：
文献类型：
--
作者：
P. Bartlett;Sébastien Bubeck;Yeshwanth Cherapanamjeri
我们考虑了具有独立高斯参数的ReLU网络中的对抗性示例现象。对于深度恒定且宽度范围很大的网络（例如，如果每层的宽度是任何其他层宽度的多项式就足够了），输入向量的小扰动会导致输出的大变化。这推广了Daniely和Schacham（2020）对宽度迅速减小的网络以及Bubeck等人（2021）对双层网络的结果。证明表明，这些网络中出现了对抗性的例子，因为它们计算的函数非常接近线性。网络中的瓶颈层起着关键作用：网络中某个点的最小宽度决定了到该点为止计算的映射的尺度和灵敏度。主要结果是针对具有恒定深度的网络，但我们也表明，对于这种结果，对深度的一些约束是必要的，因为存在适当深度的网络，以恒定的概率计算接近常数的函数。
We consider the phenomenon of adversarial examples in ReLU networks with independent gaussian parameters. For networks of constant depth and with a large range of widths (for instance, it suffices if the width of each layer is polynomial in that of any other layer), small perturbations of input vectors lead to large changes of outputs. This generalizes results of Daniely and Schacham (2020) for networks of rapidly decreasing width and of Bubeck et al (2021) for two-layer networks. The proof shows that adversarial examples arise in these networks because the functions that they compute are very close to linear. Bottleneck layers in the network play a key role: the minimal width up to some point in the network determines scales and sensitivities of mappings computed up to that point. The main result is for networks with constant depth, but we also show that some constraint on depth is necessary for a result of this kind, because there are suitably deep networks that, with constant probability, compute a function that is close to constant.