Speech Prosody and Articulatory Dynamics in Spoken Language

描述（由申请人提供）：语言的一个强有力的交际方面是韵律，韵律是使用音调、响度、局部速率调制和语音中的停顿来表示其信息和情感内容。由于神经损伤如中风或自闭症谱系障碍引起的言语产生缺陷通常以韵律不规则为特征。本研究计划的长期目标是了解语言结构和交际语境是如何制约说话过程中发音运动的时空实现的。我们的语言学家-工程师团队在发音模式的层面上研究韵律的“签名”。这个建议的具体目的是了解和建模扬声器如何不同地调制的发音手势的时空组织作为一个功能的认知源的语音流中的一个中断，以及如何的交际环境影响的语音流的时间流在发音扬声器相互作用。我们概述了一个研究策略，调查语音启动/停止和控制和协调的清晰度之间的关系。除了语言结构外，言语的开始、暂停或停止可能有多种原因 短语边缘。言语流中的一些中断可能是认知上“有计划的”，例如对话者在话语中的话轮转换。语音流中的其他中断可能是“计划外的”，例如中断和单词查找挑战。我们的方法研究了语音的发音在声道轮和中断结构化对话，并在附近的暂停发生认知语音规划的原因。我们补充这个实验工作的计算建模的短语连接和停顿，并与机器学习方法来分类中断语音产生不同的来源。具体目标将通过使用磁力计系统收集的发音运动数据来跟踪口腔内的运动，并使用我们团队的语音产生计算模型来实现。实验工作和随之而来的发音研究结果的计算建模将提供一个轮廓的方式，其中发音模式是由更大的信息结构的话语和言语规划的要求，在一个沟通的背景下。 公共卫生相关性：语言的一个强有力的交际方面是韵律，这是使用音高，响度和时间属性，如停顿在讲话中，以信号的信息和情感内容。由于神经损伤如中风或自闭症谱系障碍导致的言语产生缺陷通常以韵律不规则为特征，理解结构韵律及其在言语时间流通信中的部署的影响可以具有关键的翻译影响，因为不流利通常用作言语障碍诊断的基础。我们的研究使用仪器跟踪语音过程中的发音运动，以提供对语音流中调制和停顿的规范性生产的理解，这可以支持临床人群中韵律故障的循证评估和治疗，包括为受损者部署辅助技术，如自动语音识别和机器语音合成。

DESCRIPTION (provided by applicant): One powerfully communicative aspect of language is prosody, which is the use of pitch, loudness, local rate modulation, and pauses in speech to signal its informational and affective content. Speech production deficits due to neurological damage such as stroke or due to Autism Spectrum Disorder are often characterized by prosodic irregularities. The long term objective of the proposed research program is to understand how linguistic structure and communicative context condition the spatiotemporal realization of articulatory movement during speaking. Our linguist-engineer team studies the "signatures" of prosody at the level of articulatory patterning. The specific aims of this proposal are to understand and model how speakers differentially modulate the spatiotemporal organization of articulatory gestures as a function of the cognitive source of a break in the speech stream and how the communicative context influences the temporal flow of the speech stream in articulation as speakers interact with one another. We outline a research strategy that investigates the relation between speech initiation/cessation and the control and coordination of articulation. Speech may start, pause, or cease for a variety of reasons in addition to linguistically structured phrase edges. Some breaks in the speech stream may be cognitively "planned," such as interlocutor turn-taking in discourse. Other disruptions in the speech stream might be "unplanned," such as interruptions and word finding challenges. Our approach investigates the articulation of speech in the vocal tract at turn-taking and interruptions in structured dialogue and in the vicinity of pauses that occur for cognitive speech planning reasons. We complement this experimental work with computational modeling of phrasal junctures and pauses, and with machine learning approaches to classifying breaks in speech arising from differing sources. The specific aims will be pursued by using articulatory movement data collected with magnetometer systems for tracking movement inside the mouth and by using our team's computational model of speech production. The experiment work and the concomitant computational modeling of the articulatory findings will provide a profile of the manner in which articulatory patterning is shapd by the larger informational structuring of utterances and by the demands of speech planning in a communicative context. PUBLIC HEALTH RELEVANCE: One powerfully communicative aspect of language is prosody, which is the use of pitch, loudness, and temporal properties such as pauses in speech to signal its informational and affective content. Speech production deficits due to neurological damage such as stroke or due to Autism Spectrum Disorder are often characterized by prosodic irregularities and understanding the influence of structural prosody and its deployment in communication on the temporal flow of speech can have critical translational impact in that disfluencies are typically used as a basis for diagnosis of speech disorders. Our research uses instrumental tracking of articulatory movements during speech to provide an understanding of normative production of modulation and pauses in speech flow that could support evidence-driven assessments and treatments of prosodic breakdown in clinical populations, including deploying assistive technologies for the impaired such as automatic speech recognition and machine speech synthesis.