RI Core: Medium: Structured variability in vocal tract articulation dynamics in speech

人类产生和使用语言来相互交流和互动，以各种各样的方式传达他们的思想和表达情感。一个人产生丰富的语音涉及复杂的运动和协调的发音器官，如舌头和下巴，以灵活和适应的方式相互作用的背景。然而，如何实现这种灵活性的细节并不完全清楚。言语产生也会受到各种个人情况的影响，包括疾病和障碍。该项目将通过直接观察和建模语音中的清晰度，为理解人类语音如何随时间变化奠定科学基础，无论是在人际交往中还是在人际交往中--几小时、几天、几周、几个月和几年。这些知识对于推进语音科学和设计鲁棒的交互式语音技术都是至关重要的。言语研究的一个长期目标是理解和解决其产生中的丰富和普遍的变异性，无论是在个体内部还是在个体之间，还是在不同的干扰语境中。我们的研究调查的问题，不能通过语音声学单独接近。直接获得动态信息的声道清晰度，辅以技术和分析的进步，使我们能够检查复杂的行为与语音生产的可变性，即其灵活性和稳定性随着任务和时间的推移。该项目将使用先进的实时磁共振成像（rtMRI）和语音产生过程中人类声道运动的计算建模，以了解跨时间尺度的口语交流的结构和控制，无论是在个人经验还是在人际互动中，提供了一个前所未有的机会，观察人类如何计划和生产语音协作与另一个时空的细节，可能之前。该项目的创新包括在两个地点同时和同步地对对话扬声器的声道进行成像，以了解对话期间的语音产生行为，映射单个个体的生产语音生产如何自然变化，通常在几个小时，几天，几个月和几年内，以及语音灵活性的个体差异如何预测扬声器稳定性。该研究计划通过利用丰富的，定量的和动态的发音rtMRI数据的实证工作整合了语音科学和工程，并将广泛分享独特的数据，工具和模型。该项目还具有语音技术以外的重要应用意义，因为对规范性发音及其变异性的了解可以通过帮助获得强健的言语来影响言语障碍的评估和补救，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值进行评估来支持和更广泛的影响审查标准。

Humans produce and use speech to communicate and interact with one another, to convey their thoughts and to their express emotions, in a vast variety of ways. The production of the rich sounds of speech by an individual involves intricate movement and coordination of the vocal organs, such as the tongue and jaw, in a manner that is flexible and adaptive to the context of the interaction. Yet, the details of how this flexibility is achieved are not known completely. Speech production can also be affected by a variety of personal circumstances including illness and disorder. The project will create a scientific foundation for understanding how human speech varies across time, both within and across interpersonal interactions––over hours, days, weeks, months and years––by directly observing and modeling articulation during speech. Such knowledge is fundamental to both advancing speech science and to the design of robust interactive speech technologies. A longstanding goal in speech research is to understand and address the rich and pervasive variability in its production, both within and across individuals and for varied interactional contexts. Our research investigates questions not approachable via speech acoustics alone. Direct access to dynamic information on vocal tract articulation, complemented by technology and analysis advances, allow us to examine complex behavior associated with speech production variability—namely, its flexibility and stability over task and over time. The project will use advanced real-time magnetic resonance imaging (rtMRI) and computational modeling of the human vocal tract motion during speech production to understand the structure and control of spoken language communication across timescales, both within individual experience and across interpersonal interactions, offering an unprecedented opportunity to observe how humans plan and produce speech collaboratively with one another at a spatiotemporal detail not possible before. The project innovations include imaging the vocal tracts of conversing speakers simultaneously and synchronously at two sites to understand speech production behavior during a dialog, mapping how a single individual’s production speech production varies naturally and typically over hours, days, months and years, and how individual differences in speech flexibility are predictive of speaker stability. The research program integrates speech science and engineering through empirical work leveraging rich, quantitative, and dynamic articulatory rtMRI data, and will broadly share the unique data, tools and models. The project also has critical applied significance beyond speech technology, as knowledge of normative articulation and its variability can impact the assessment and remediation of speech disorders by helping derive robust speech-based biomarkers for a variety of clinical conditions across the life span from Autism to dementia.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.