Doctoral Dissertation Research:Modeling temporal coordination in speech production using an artificial central pattern generator neural network

这个项目研究了语音生产中的时间是如何协调的，以及这种协调如何使用人工神经网络进行计算建模。研究表明，语音节奏反映了语音单位（音节、重音单位、短语）的多层次、分层组织，这些单位本质上是周期性的。一种具有这些特性的神经网络，即中央模式发生器（CPG），已经被假设为语音定时的基础。本项目提出了两个语音生成实验，旨在研究法语和英语这两种具有不同节奏类型的语言的三个语音单元内部和之间的时间协调，并发现它们共享哪些时间属性，哪些是特定于语言的。本文提出了一种三层cpg型人工神经网络，对两个实验的结果进行了建模，以检验该模型模拟两种节奏不同的语言的时序行为的能力。实验1通过比较两种语言中短语重音延长和短语末音节延长的持续效应，关注短语、重音单位和音节之间的协调。实验2通过测量重复多次的短语发音时间的变化，比较两种语言中口语短语及其组成部分的整体时间协调性。所提出的研究和建模工作的主要目标是调查语音单位层次结构之间的时间协调，特别是语言之间的协调如何变化，并测试生物学启发的cpg型神经网络模型对这种协调和由此产生的语音模式进行建模的能力。这项工作有助于理解语言中节奏单位的协调和定时，以及大脑如何产生这种协调。就其更广泛的影响而言，它将使语言产生理论与生物和认知模型中的其他运动行为理论相结合，并有助于在语言、语言韵律和更广泛的动物行为神经模型研究之间建立联系。这项工作也将有助于建立一个统一的节奏和韵律模型，足以解释潜在的普遍性和人类语言的变化。该项目还将丰富研究生共同项目负责人的培养。

This project investigates how timing is coordinated in speech production and how this coordination can be computationally modeled using artificial neural networks. Research has shown that speech rhythm reflects a multi-tiered, hierarchical organization of speech units (syllables, accentual units, phrases) that are essentially cyclical in nature. One type of neural network which shares these properties, the central pattern generator (CPG), has been hypothesized to underlie speech timing. This project presents two speech production experiments designed to investigate temporal coordination within and among three levels of speech units in French and English, languages with distinct rhythm types, and to discover which timing properties they share and which are language-specific. A three-level CPG-type artificial neural network is presented which is used to model the results of both experiments in order to test the ability of such a model to simulate the timing behavior of two rhythmically distinct languages. Experiment 1 focuses on the coordination between phrases, accentual units, and syllables through a comparison of the durational effects of lengthening due to phrasal stress and phrase-final syllable lengthening in both languages. Experiment 2 compares the overall temporal coordination of spoken phrases and their constituent parts in the two languages by measuring the changes in articulatory timing of phrases which are repeated multiple times. The main goals of the proposed studies and modeling work are to investigate temporal coordination between the hierarchically structured levels of speech units, specifically how that coordination varies between languages, and to test the ability of a biologically inspired CPG-type neural network model to model this coordination and the speech patterns that result from it.This work contributes to an understanding of the coordination and timing of rhythm units in speech and how that coordination may be generated by the brain. In its broader impact, it will enable the integration of theories of speech production with other motor behavior in biological and cognitive models, and help to forge links between research on speech, linguistic prosody, and more broadly, neural models of animal behavior. This work will also contribute to the construction of a unified model of rhythm and prosody which is sufficient to explain both the underlying universalities and the variations of human language. The project will also enrich the training of the graduate student co-PI.