Characterizing the temporal processing of speech in the human auditory cortex

表征人类听觉皮层中语音的时间处理

• 批准号: 10626815
• 负责人: Adeen Flinker
• 金额: $ 66.12万
• 依托单位: NEW YORK UNIVERSITY SCHOOL OF MEDICINE
• 依托单位国家: 美国
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10626815
• 关键词: Acoustic Nerve; Acoustics; Address; Anatomy; Area; Auditory; Auditory area; Behavioral; Behavioral Research; Brain; Categories; Communication; Communication impairment; Communities; Complex; Computer Models; Data; Dimensions; Discipline; Early identification; Focal Brain Injuries; Heterogeneity; Human; Impairment; Implanted Electrodes; Language Disorders; Linguistics; Link; Maps; Measures; Methods; Modeling; Nature; Neural Network Simulation; Neurobiology; New York; Patients; Pattern; Phonetics; Process; Prosthesis; Research; Research Personnel; Resolution; Route; Scheme; Semantics; Signal Transduction; Speech; Speech Disorders; Speech Pathology; Speech Perception; Speech Sound; Stimulus; Structure; Surface; System; Techniques; Technology; Testing; Time; Universities; auditory pathway; behavior observation; cohort; deep neural network; density; design; experimental study; human model; interest; learning network; lexical; millisecond; neural; neuroimaging; neurophysiology; neurosurgery; novel; novel therapeutic intervention; pandemic disease; phonology; phrases; receptive field; response; sound; spatiotemporal; speech processing; speech recognition; syntax; temporal measurement

Project Summary Time is the fundamental dimension of sound, and temporal integration is thus fundamental to speech perception. To recognize a complex structure such as a word in fluent speech, the brain must integrate across many different timescales spanning tens to hundreds of milliseconds. These timescales are considerably longer than the duration of responses at the auditory nerve. Therefore, the auditory cortex must integrate acoustic information over long and varied timescales to encode linguistic units. On the other hand, the nature of the intermediate units of representation between sound and meaning remains debated. Focal brain injuries have shown selective impairment at all levels of linguistic processing (phonemic, phonotactic, and semantic) but current models of spoken word recognition disagree on the existence and type of these representational levels. The neural basis of temporal and linguistic processing remains speculative partly due to the limited spatiotemporal resolution of noninvasive human neuroimaging techniques which is needed to study the encoding of fluent speech. Our multi- PI proposal overcomes these challenges by assembling a team of researchers and clinicians with complementary expertise at NYU and Columbia University. We propose to record invasively from a large number of neurosurgical patients, which provides a rare and unique opportunity to collect direct cortical recordings across several auditory regions. We propose novel experimental paradigms and analysis methods to investigate where, when, and how acoustic features of speech are integrated over time to encode linguistic units. Our experimental paradigms will determine the functional and anatomical organization of stimulus integration periods in primary and nonprimary auditory cortical regions and relate the temporal processing in these regions to the emergence of phonemic-, phonotactic-, and semantic-level representations. Finally, we will determine the nonlinear computational mechanisms that enable the auditory cortex to integrate fast features over long durations, which is essential in speech recognition. Understanding of the temporal processing of speech in primary and nonprimary auditory cortex is critical for developing complete models of speech perception in the human brain, which is essential to understanding of how these processes break down in speech and communication disorders.

项目摘要 时间是声音的基本维度，因此时间整合是言语感知的基础。 为了识别一个复杂的结构，如流利的演讲中的一个单词，大脑必须整合许多不同的 时间跨度从几十毫秒到数百毫秒。这些时间尺度比 听神经反应的持续时间。因此，听觉皮质必须整合声学信息 在漫长而多样的时间尺度上对语言单位进行编码。另一方面，中间单位的性质 声音和意义之间的表征仍然存在争议。局灶性脑损伤显示出选择性 所有语言加工水平的障碍(音素、音位和语义)，但目前的模型 口语识别对这些表征水平的存在和类型存在分歧。神经基础 仍然是推测的，部分原因是时间和语言处理的时空分辨率有限 研究流利语音编码所需的非侵入性人体神经成像技术。我们的多- PI提案通过将一组研究人员和临床医生与 纽约大学和哥伦比亚大学的互补专业知识。我们建议从大量数据中进行侵入性记录 神经外科患者，这提供了一个难得而独特的机会来收集跨 几个听觉区域。我们提出了新的实验范式和分析方法来研究， 语音的声学特征何时以及如何随时间被整合以编码语言单位。我们的实验 范式将决定小学刺激整合期的功能和解剖组织 和非初级听觉皮质区域，并将这些区域的时间处理与出现联系起来 音素、音位和语义级的表征。最后，我们将确定非线性 计算机制，使听觉皮质能够在较长时间内整合快速功能，这 在语音识别中是必不可少的。小学生和小学生对言语时间加工的理解 非初级听觉皮质对于在人脑中建立完整的语音感知模型至关重要， 这对于理解这些过程是如何在言语和交流障碍中分解是至关重要的。

项目成果

Deep neural networks effectively model neural adaptation to changing background noise and suggest nonlinear noise filtering methods in auditory cortex.

• DOI: 10.1016/j.neuroimage.2022.119819
• 发表时间: 2023-02-01
• 期刊: NEUROIMAGE
• 影响因子: 5.7
• 作者: Mischler, Gavin;Keshishian, Menoua;Bickel, Stephan;Mehta, Ashesh D.;Mesgarani, Nima
• 通讯作者: Mesgarani, Nima

Distinct neural encoding of glimpsed and masked speech in multitalker situations.

• DOI: 10.1371/journal.pbio.3002128
• 发表时间: 2023-06
• 期刊: PLoS biology
• 影响因子: 9.8

Timing and location of speech errors induced by direct cortical stimulation.

直接皮层刺激引起的言语错误的时间和位置。

• DOI: 10.1101/2023.09.14.557732
• 发表时间: 2024
• 期刊: bioRxiv : the preprint server for biology
• 作者: Kabakoff,Heather;Yu,Leyao;Friedman,Daniel;Dugan,Patricia;Doyle,WernerK;Devinsky,Orrin;Flinker,Adeen
• 通讯作者: Flinker,Adeen

Interaction of bottom-up and top-down neural mechanisms in spatial multi-talker speech perception.

空间多说话者语音感知中自下而上和自上而下神经机制的相互作用。

• DOI: 10.1016/j.cub.2022.07.047
• 发表时间: 2022
• 期刊: Current biology : CB
• 作者: Patel,Prachi;vanderHeijden,Kiki;Bickel,Stephan;Herrero,JoseL;Mehta,AsheshD;Mesgarani,Nima
• 通讯作者: Mesgarani,Nima

naplib-python: Neural acoustic data processing and analysis tools in python.

• DOI: pii: 100541. doi: 10.1016/j.simpa.2023.100541
• 发表时间: 2023-09
• 期刊: Software impacts
• 作者: Mischler G;Raghavan V;Keshishian M;Mesgarani N
• 通讯作者: Mesgarani N