SYNAPTIC DEPRESSION IN DEEP NEURAL NETWORKS FOR SPEECH PROCESSING.
SYNAPTIC DEPRESSION IN DEEP NEURAL NETWORKS FOR SPEECH PROCESSING.
复制标题
DOI：
10.1109/icassp.2016.7472802
复制
发表时间：
2016-03
期刊：
影响因子：
--
通讯作者：
Mesgarani N
中科院分区：
文献类型：
--
作者：
Zhang W;Li H;Yang M;Mesgarani N
生物神经元的一个特征是它们能够动态地改变突触效能以响应不同的输入条件。这种机制被称为突触抑制，对言语特征的规范化表征的形成有重要作用。突触抑制也有助于生物系统的健壮性能。在本文中，我们描述了如何将突触抑制建模并纳入深度神经网络架构以提高其泛化能力。我们观察到，当突触抑制被添加到神经网络的隐藏层时，它减少了改变背景活动对节点激活的影响。此外，我们表明，当将突触抑制包含在用于音素分类的深度神经网络中时，网络在未包含在训练阶段的噪声条件下的性能得到改善。我们的研究结果表明，更完整的神经元模型可以进一步缩小生物性能和人工计算之间的差距，从而使网络更好地泛化到新的信号条件。
A characteristic property of biological neurons is their ability to dynamically change the synaptic efficacy in response to variable input conditions. This mechanism, known as synaptic depression, significantly contributes to the formation of normalized representation of speech features. Synaptic depression also contributes to the robust performance of biological systems. In this paper, we describe how synaptic depression can be modeled and incorporated into deep neural network architectures to improve their generalization ability. We observed that when synaptic depression is added to the hidden layers of a neural network, it reduces the effect of changing background activity in the node activations. In addition, we show that when synaptic depression is included in a deep neural network trained for phoneme classification, the performance of the network improves under noisy conditions not included in the training phase. Our results suggest that more complete neuron models may further reduce the gap between the biological performance and artificial computing, resulting in networks that better generalize to novel signal conditions.