BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization.
BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization.
复制标题
DOI：
10.3389/fnbot.2021.567482
复制
发表时间：
2021
影响因子：
3.1
通讯作者：
Zhao L
中科院分区：
文献类型：
--
作者：
Gao Y;Ascoli GA;Zhao L
深度神经网络（DNN）以从大量数据中提取有用信息而闻名。然而，在DNN中学习的表示通常很难解释，特别是在密集层中。经典DNN模型（如多层感知器（MLP））的一个关键问题是，DNN同一层中的神经元相互之间是有条件独立的，这使得协同训练和出现更高的模块性变得困难。与DNN相反，哺乳动物大脑中的生物神经元显示出大量的依赖模式。具体来说，生物神经网络通过所谓的神经元组装来编码表示：通过强突触相互作用互连并共享联合语义内容的神经元组。由此产生的群体编码对于人类认知和记忆过程至关重要。在这里，我们提出了一种新的生物增强人工神经元组装（BEAN）正则化1来模拟神经元的相关性和依赖性，灵感来自神经科学的细胞组装理论。实验结果表明，BEAN能够形成可解释的神经元功能簇，从而促进稀疏，内存/计算效率的网络，而不会损失模型性能。此外，我们的少量学习实验表明，BEAN也可以提高模型的泛化能力，当训练样本非常有限时。
Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization1 to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited.
登录
查看更多内容
影响因子：
1.2
作者：
Peyrache A;Benchenane K;Khamassi M;Wiener SI;Battaglia FP
通讯作者：
Battaglia FP
影响因子：
15.9
作者：
Rees CL;Moradi K;Ascoli GA
通讯作者：
Ascoli GA
影响因子：
1.6
作者：
Ascoli, GA;Atkeson, JC
通讯作者：
Atkeson, JC
影响因子：
4.4
作者：
GRANOVETTER, MS
通讯作者：
GRANOVETTER, MS
影响因子：
1.8
作者：
DEVALOIS, RL;YUND, EW;HEPLER, N
通讯作者：
HEPLER, N