On 1/n neural representation and robustness
On 1/n neural representation and robustness
复制标题
DOI：
--
复制
发表时间：
2020-12
期刊：
影响因子：
--
通讯作者：
Josue Nassar;Piotr A. Sokól;SueYeon Chung;K. Harris;Il Memming Park
中科院分区：
文献类型：
--
作者：
Josue Nassar;Piotr A. Sokól;SueYeon Chung;K. Harris;Il Memming Park
理解神经网络表征的本质是神经科学和机器学习的共同目标。因此，令人兴奋的是，这两个领域不仅在共同的问题上融合，而且在相似的方法上融合。在这些领域中，一个紧迫的问题是理解神经网络使用的表示结构如何影响它们的泛化和对扰动的鲁棒性。在这项工作中，我们通过将有关小鼠V1 （Stringer等人）与人工神经网络中神经表征的协方差谱的实验结果并列来研究后者。我们使用对抗鲁棒性来探索Stringer等人关于1/n协方差谱因果作用的理论。我们实证研究了这种神经代码在神经网络中所带来的好处，并阐明了它在多层体系结构中的作用。我们的研究结果表明，将实验观察到的结构强加于人工神经网络使其对对抗性攻击更具鲁棒性。此外，我们的发现通过显示中间表征的作用，补充了将宽神经网络与核方法相关的现有理论。
Understanding the nature of representation in neural networks is a goal shared by neuroscience and machine learning. It is therefore exciting that both fields converge not only on shared questions but also on similar approaches. A pressing question in these areas is understanding how the structure of the representation used by neural networks affects both their generalization, and robustness to perturbations. In this work, we investigate the latter by juxtaposing experimental results regarding the covariance spectrum of neural representations in the mouse V1 (Stringer et al) with artificial neural networks. We use adversarial robustness to probe Stringer et al's theory regarding the causal role of a 1/n covariance spectrum. We empirically investigate the benefits such a neural code confers in neural networks, and illuminate its role in multi-layer architectures. Our results show that imposing the experimentally observed structure on artificial neural networks makes them more robust to adversarial attacks. Moreover, our findings complement the existing theory relating wide neural networks to kernel methods, by showing the role of intermediate representations.