Doctoral Dissertation Research: Articulatory Dynamics and Stability of Multi-gesture Complexes

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。口语的产生涉及舌头、嘴唇、鼻口和喉部的联合和协调的发音动作；然而，语言的自然变化和“看到”人类声道内发生的事件的困难意味着，这些对口语形成至关重要的发音事件的优雅编排仍然是一个谜。这个项目是一项使用实时磁共振成像（rtMRI）来研究语音产生的技术，这种技术可以在讲话时对声道运动进行成像。该项目的目标是描述发音动作的动态特征，因为它们被组织成协调的复合体，如语音片段（大约“字母大小”的单位）。对协调稳定性的研究将有助于科学家理解用于构建和说出我们所知道的单词的语言单位的认知表征。最终，这种理解对语言崩溃（例如，由于中风或疾病）、语言习得、语言和方言多样性以及合成和识别人类语言的技术都有影响。本论文对语音产生和伴随的计算建模进行实时MRI实验，以研究语音片段的最小运动“原子”如何系统地协调。该项目测试了声道运动组成一个节段单元是紧密协调的假设。因此，他们的协调被预测为在自然语境变化的语言中是稳定的，如短语边界和强调，不像其他跨越片段的运动序列，它们表现出更多的可塑性协调。利用两种语言的数据，对声门辅音的喉-口协调和鼻辅音的喉-口协调进行动态成像，并跟踪声道运动以分析空间和时间上的发音模式和协调。使用动态耦合模型对经验发现进行了计算评估，其中稳定性程度可以从某些耦合结构中出现。该项目旨在将协调关系纳入语言认知表征，从而阐明构成单词的音位单位编码系统中语音片段的动态和突现性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Spoken language production involves combined and coordinated articulatory actions of the tongue, lips, nasal port, and larynx; yet the natural variation in speech and the difficulty of "seeing" events occurring inside the human vocal tract mean that the elegant choreography of these articulatory events critical to the formation of spoken words remains mysterious. This project is a study of speech production using real-time Magnetic Resonance Imaging (rtMRI)—a technique that can image vocal tract movement during speech. The project's goal is to characterize the dynamics of articulatory actions as they are organized into coordinated complexes such as speech segments (roughly ‘alphabetic-sized’ units). The investigation of stabilities in coordination will help scientists understand the cognitive representation of the linguistic units that are used to structure and say the words we know. Ultimately such an understanding has implications for language in breakdown (e.g., due to stoke or disease), for language acquisition, for language and dialect diversity, and for technologies that synthesize and recognize human speech.This dissertation undertakes real-time MRI experiments on speech production and concomitant computational modeling to examine how the smallest movement 'atoms' of speech segments are systematically coordinated. The project tests the hypothesis that vocal tract movements composing a segmental unit are tightly coordinated with one another. Therefore, their coordination is predicted to be stable across natural contextual variations in speech such as phrase boundaries and emphasis, unlike other movement sequences spanning across segments, which have been shown to exhibit more plastic coordination. Using data from two under-studied languages, both larynx-oral coordination in glottalic consonants and velum-oral coordination in nasal consonants are dynamically imaged, and vocal tract movements are tracked to analyze articulatory patterning and coordination in space and time. Empirical findings are computationally assessed using a model of dynamical coupling with which degrees of stability can emerge from certain coupling structures. The project seeks to incorporate coordination relations into linguistic cognitive representations and thereby illuminate the dynamic and emergent nature of speech segments in the encoding system of the phonological units that form words.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.