Evidence of a predictive coding hierarchy in the human brain listening to speech.
Evidence of a predictive coding hierarchy in the human brain listening to speech.
复制标题
DOI：
10.1038/s41562-022-01516-2
复制
发表时间：
2023-03
影响因子：
29.9
通讯作者：
King, Jean-Remi
中科院分区：
文献类型：
--
作者：
Caucheteux, Charlotte;Gramfort, Alexandre;King, Jean-Remi
最近，自然语言处理取得了长足的进步：深度学习算法越来越能够生成、总结、翻译和分类文本。然而，这些语言模型仍然无法匹配人类的语言能力。预测编码理论对这种差异提供了一个初步的解释：虽然语言模型经过优化可以预测附近的单词，但人脑会持续预测跨越多个时间尺度的表示层次结构。为了验证这一假设，我们分析了 304 名听短故事的参与者的功能磁共振成像大脑信号。首先，我们确认现代语言模型的激活线性映射到大脑对语音的反应。其次，我们表明，通过跨越多个时间尺度的预测来增强这些算法可以改善这种大脑映射。最后，我们表明这些预测是分层组织的：额顶叶皮层比颞叶皮层预测更高级别、更远范围和更多上下文的表征。总体而言，这些结果加强了分层预测编码在语言处理中的作用，并说明了神经科学和人工智能之间的协同作用如何揭示人类认知的计算基础。当前的机器学习语言算法会进行相邻单词级别的预测。在这项工作中，Caucheteux 等人。表明人类大脑可能使用长期和分层的预测，考虑到未来最多八个可能的单词。
Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these language models still fail to match the language abilities of humans. Predictive coding theory offers a tentative explanation to this discrepancy: while language models are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. To test this hypothesis, we analysed the functional magnetic resonance imaging brain signals of 304 participants listening to short stories. First, we confirmed that the activations of modern language models linearly map onto the brain responses to speech. Second, we showed that enhancing these algorithms with predictions that span multiple timescales improves this brain mapping. Finally, we showed that these predictions are organized hierarchically: frontoparietal cortices predict higher-level, longer-range and more contextual representations than temporal cortices. Overall, these results strengthen the role of hierarchical predictive coding in language processing and illustrate how the synergy between neuroscience and artificial intelligence can unravel the computational bases of human cognition. Current machine learning language algorithms make adjacent word-level predictions. In this work, Caucheteux et al. show that the human brain probably uses long-range and hierarchical predictions, taking into account up to eight possible words into the future.
登录
查看更多内容
影响因子：
25
作者：
Goldstein A;Zada Z;Buchnik E;Schain M;Price A;Aubrey B;Nastase SA;Feder A;Emanuel D;Cohen A;Jansen A;Gazula H;Choe G;Rao A;Kim C;Casto C;Fanda L;Doyle W;Friedman D;Dugan P;Melloni L;Reichart R;Devore S;Flinker A;Hasenfratz L;Levy O;Hassidim A;Brenner M;Matias Y;Norman KA;Devinsky O;Hasson U
通讯作者：
Hasson U
影响因子：
4.3
作者：
Gramfort A;Luessi M;Larson E;Engemann DA;Strohmeier D;Brodbeck C;Goj R;Jas M;Brooks T;Parkkonen L;Hämäläinen M
通讯作者：
Hämäläinen M
DOI：
10.1073/pnas.1612132113
发表时间：
2016-10-11
影响因子：
11.1
作者：
Fedorenko, Evelina;Scott, Terri L.;Kanwisher, Nancy
通讯作者：
Kanwisher, Nancy
影响因子：
3.7
作者：
Felleman, Daniel J.;Van Essen, David C.
通讯作者：
Van Essen, David C.
影响因子：
48
作者：
Esteban, Oscar;Markiewicz, Christopher J.;Gorgolewski, Krzysztof J.
通讯作者：
Gorgolewski, Krzysztof J.