Collaborative Research: RI: Small: From Ultrasound and MRI to articulatory and acoustic models of child speech development

合作研究：RI：小型：从超声和 MRI 到儿童言语发展的发音和声学模型

• 批准号: 2006979
• 负责人: Abeer Alwan
• 金额: $ 23万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2006979&HistoricalAwards=false
• 关键词: Collaborative; Research; RI; Small; Ultrasound

Scientific models of adult speech matured several decades ago and have proven extremely useful in speech technologies. In contrast, models of child speech remain poorly motivated and grounded in adult rather than child data, in part because of the difficulties involved in collecting speech articulation data from children. The lack of appropriate models for child speech is increasingly problematic as technological application areas for children emerge and become more important. The project will advance the scientific and technological state-of-the-art related to child speech to develop a variety of applications such as speech-based interactive systems for literacy development and learning.Scientifically, the project will 1) reveal relationships between articulation and acoustics in child speech, 2) characterize previously unobserved processes of speech development in children, including anatomical development of the vocal tract, and 3) develop acoustic models and speech technologies for child speech, which can be scalable with age. The different aspects of the project are synergistic: findings from articulation and acoustic experiments will inform the development of algorithms essential to speech technologies for children. Key strengths of the project are the 1) uniqueness of the data corpus that will be created and leveraged, and 2) interdisciplinary approach to studying speech production and developing models of child speech for use in speech technologies. Production data will include 3D/4D ultrasound recordings of the tongue, MRI anatomical scans of the head and neck, palate impressions, and microphone recordings. The corpus will, for the first time, allow the relationship between articulatory and acoustic variability, and the longitudinal development of speech articulation, to be directly explored and modeled for child speech. The project will advance knowledge of articulatory-acoustic relations and their development in children's speech, and their relationship to machine recognition of children's speech.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

成人语音的科学模型在几十年前就已经成熟，并在语音技术中被证明非常有用。相比之下，儿童语音模型仍然动机不佳，并在成人而不是儿童数据的基础上，部分原因是在收集儿童语音清晰度数据的困难。随着针对儿童的技术应用领域的出现和变得越来越重要，缺乏适当的儿童语音模型越来越成问题。该项目将推进与儿童语言有关的科学和技术发展水平，以开发各种应用，如用于识字发展和学习的基于语言的互动系统。该项目将1）揭示儿童语言中的清晰度和声学之间的关系，2）描述以前未观察到的儿童语言发展过程，包括声道的解剖学发展，以及3）开发用于儿童语音的声学模型和语音技术，其可以随年龄而扩展。该项目的不同方面是协同的：发音和声学实验的结果将为儿童语音技术所必需的算法的开发提供信息。 该项目的主要优势是：1）将创建和利用的数据语料库的独特性，以及2）研究语音生成和开发用于语音技术的儿童语音模型的跨学科方法。 生产数据将包括舌头的3D/4D超声记录，头部和颈部的MRI解剖扫描，腭印象和麦克风记录。该语料库将首次允许发音和声学变化之间的关系，以及语音发音的纵向发展，直接探索和建模儿童语音。该项目将促进对儿童语音中发音-声学关系及其发展的了解，以及它们与儿童语音机器识别的关系。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Fundamental Frequency Feature Normalization and Data Augmentation for Child Speech Recognition

• DOI: 10.1109/icassp39728.2021.9413801
• 发表时间: 2021-02
• 期刊: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Gary Yeung;Ruchao Fan;A. Alwan

Development of vowel acoustics and subglottal resonances in American English-speaking children: A longitudinal Study

美国英语儿童元音声学和声门下共振的发展：一项纵向研究

• DOI: 10.1121/10.0016294
• 发表时间: 2022
• 期刊: The Journal of the Acoustical Society of America
• 作者: Shetty, Vishwas;Lulich, Steven M.;Palo, Pertti;Alwan, Abeer
• 通讯作者: Alwan, Abeer

Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-Training and its Application to Children’s ASR

• DOI: 10.1109/icassp39728.2021.9414970
• 发表时间: 2021-02
• 期刊: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Ruchao Fan;Amber Afshan;A. Alwan

Developmental Articulatory and Acoustic Features for Six to Ten Year Old Children

• DOI: 10.21437/interspeech.2023-2236
• 发表时间: 2023-08
• 作者: Vishwas M. Shetty;Steven M. Lulich;A. Alwan

Fundamental frequency feature warping for frequency normalization and data augmentation in child automatic speech recognition

• DOI: 10.1016/j.specom.2021.08.002
• 发表时间: 2021-12
• 期刊: Speech Commun.
• 作者: Gary Yeung;Ruchao Fan;A. Alwan