Dissecting neural computations in the human auditory pathway using deep neural networks for speech.
Dissecting neural computations in the human auditory pathway using deep neural networks for speech.
复制标题
使用语音深度神经网络剖析人类听觉通路中的神经计算。
DOI：
10.1038/s41593-023-01468-4
复制
发表时间：
2023-12
影响因子：
25
通讯作者：
Chang, Edward F.
中科院分区：
文献类型：
--
作者：
Li, Yuanning;Anumanchipalli, Gopala K.;Mohamed, Abdelrahman;Chen, Peili;Carney, Laurel H.;Lu, Junfeng;Wu, Jinsong;Chang, Edward F.
人类听觉系统从语音信号中提取丰富的语言抽象。理解这一复杂过程的传统方法使用线性特征编码模型，但成功有限。人工神经网络在语音识别任务中表现出色，并提供了有前途的语音处理计算模型。我们在最先进的深度神经网络（DNN）模型中使用语音表示来研究从听觉神经到语音皮层的神经编码。DNN的分层表示与整个上行听觉系统的神经活动相关。无监督语音模型的表现至少与其他纯监督或微调模型一样好。深层DNN层与高阶听觉皮层的神经活动相关性更好，计算与语音中的音素和音节结构一致。因此，在英语或普通话上训练的DNN模型预测了每种语言的母语者的皮层反应。这些结果揭示了DNN模型表示和生物听觉通路之间的收敛，为听觉皮层中的神经编码建模提供了新的方法。利用直接颅内记录和现代语音AI模型，Li及其同事展示了用于自我监督语音学习的深度神经网络与人类听觉通路之间的代表性和计算相似性。
The human auditory system extracts rich linguistic abstractions from speech signals. Traditional approaches to understanding this complex process have used linear feature-encoding models, with limited success. Artificial neural networks excel in speech recognition tasks and offer promising computational models of speech processing. We used speech representations in state-of-the-art deep neural network (DNN) models to investigate neural coding from the auditory nerve to the speech cortex. Representations in hierarchical layers of the DNN correlated well with the neural activity throughout the ascending auditory system. Unsupervised speech models performed at least as well as other purely supervised or fine-tuned models. Deeper DNN layers were better correlated with the neural activity in the higher-order auditory cortex, with computations aligned with phonemic and syllabic structures in speech. Accordingly, DNN models trained on either English or Mandarin predicted cortical responses in native speakers of each language. These results reveal convergence between DNN model representations and the biological auditory pathway, offering new approaches for modeling neural coding in the auditory cortex. Using direct intracranial recordings and modern speech AI models, Li and colleagues show representational and computational similarities between deep neural networks for self-supervised speech learning and the human auditory pathway.
登录
查看更多内容
影响因子：
3.4
作者：
Carney LH;Li T;McDonough JM
通讯作者：
McDonough JM
影响因子：
25
作者：
Goldstein A;Zada Z;Buchnik E;Schain M;Price A;Aubrey B;Nastase SA;Feder A;Emanuel D;Cohen A;Jansen A;Gazula H;Choe G;Rao A;Kim C;Casto C;Fanda L;Doyle W;Friedman D;Dugan P;Melloni L;Reichart R;Devore S;Flinker A;Hasenfratz L;Levy O;Hassidim A;Brenner M;Matias Y;Norman KA;Devinsky O;Hasson U
通讯作者：
Hasson U
影响因子：
1.7
作者：
Carney, Laurel H.;McDonough, Joyce M.
通讯作者：
McDonough, Joyce M.
影响因子：
24.8
作者：
Bhaya-Grossman, Ilina;Chang, Edward F.
通讯作者：
Chang, Edward F.
影响因子：
4.3
作者：
Berezutskaya, Julia;Freudenburg, Zachary V.;Ramsey, Nick F.
通讯作者：
Ramsey, Nick F.