EAGER: Collaborative Research: Models of Child Speech

成人语音的产生、建模和机器识别已经研究了几十年，而儿童语音的产生、声学建模和识别却没有得到同样程度的关注。随着儿童言语和语言发展的新应用变得越来越重要和普遍，缺乏处理儿童言语问题的学术资源是有问题的。对于小学生来说尤其如此。随着孩子的成长，他们的发音也随之增长，导致他们的语音发生变化。例如，一个6岁的孩子说‘Sunny’这个词时的波形可能与同一个孩子9岁时说的完全不同。这就是为什么当前对儿童语音的机器识别在幼儿身上表现不佳，而且不能随着儿童的成长而扩大。也就是说，这些系统往往与年龄有关。随着儿童的成长，理解和模拟儿童的语音不仅对于开发更好的识别系统，而且对于更好地理解和诊断语音-语言病理(SLP)都是重要的。随着儿童早期发展缺陷的长期负面影响得到越来越广泛的认识，针对幼儿的社会和保健服务和技术应用的机会和需求也在不断增加。特别是，人们现在了解到，语言发展和识字的早期缺陷一直持续到成年，公立学校对SLP服务的需求远远超过了供应。因此，由临床医生和教师为每个儿童提供最有效的治疗或必要的关注已不再可行。更好的语音识别系统将为改善诊断和更密集的基于计算机的治疗提供机会。这个早期的探索性研究拨款项目旨在模拟语言和语言在小学期间是如何发展的，以及语言障碍儿童在语音发音方面的不同之处。儿童语音模型将导致可用于教育和治疗目的的计算机程序的开发。从科学上讲，该探索性项目将1)通过发音和声学分析的独特组合来揭示20-26岁小学生的语音产生发展过程，2)开发儿童语音的声学模型并最终开发可随年龄扩展的自动语音识别系统(相对于依赖于年龄的系统)。这只能通过了解发音和相应的声学如何随年龄发展才能实现。因此，该项目的不同方面是协同的：发音和声学实验的结果将为开发对自动语音识别至关重要的算法提供信息。生产数据将包括舌头的实时3D超声记录、嘴唇的视频记录、腭部印象、麦克风记录和颈部皮肤振动的加速度计记录，这些已被证明在自动语音和说话人识别应用中是有益的。我们将探讨发音和声学可变性之间的因果关系，以及它们与儿童语音错误识别的关系。发音功能将与声学功能一起纳入新的自动语音识别系统。这一探索性项目将有助于了解儿童之间的可变性以及儿童成长过程中随时间变化的可变性，并将首次提供规范性数据和科学模型。这些可以导致健壮的儿童语音识别系统以及将用于各种应用的工具，例如教育游戏、语音语言病理学家的培训、自动或半自动转录系统以及语音清晰度可视化系统。它将在具有技术和科学意义的重要跨学科活动中培训本科生和研究生。我们认为，拟议的项目在促进与儿童语言有关的科学和技术水平方面具有变革性。

In contrast to the production, modeling, and machine recognition of adult speech, which have been studied for decades, the production, acoustic modeling and recognition of child speech have not received the same level of attention. The lack of scholarly resources for dealing with children's speech is problematic as new applications for child speech and language development become increasingly important and commonplace. This is especially true for elementary school children. As children grow, their articulators grow as well, resulting in variations in their speech sounds. For example, the waveform of the word 'sunny' spoken by a 6-year old can be quite different than that of the same child when she is 9 years old. This is why current machine recognition of children's speech does not perform well for young children and does not scale up as the child grows. That is, these systems tend to be age dependent. Understanding and modeling child speech as children grow is important not only to developing better recognition systems but also for better understanding and diagnosis of speech-language pathology (SLP). As the negative long-term ramifications of deficits in early childhood development gain increasingly broad recognition, the opportunities and the need for social and health services and technological applications targeted toward young children are growing. In particular, it is now understood that early deficits in language development and literacy persist into adulthood, and the demand for SLP services in public schools is significantly outpacing supply. As a result, it is no longer feasible for clinicians and teachers to provide the most effective treatments or the necessary attention to every child. Better speech recognition systems would provide an opportunity for improved diagnosis and more intense computer-based therapy. This Early Grant for Exploratory Research project aims to model how speech and language develop during elementary school and how children with speech disorders differ in their articulation of speech sounds. Models of child speech will lead to the development of computer programs which can be used for educational as well as therapeutic purposes.Scientifically, the exploratory project will 1) reveal processes of speech production development in 20-26 elementary school-aged children through a unique combination of articulatory and acoustic analyses, and 2) develop acoustic models and eventually automatic speech recognition systems for children's speech which can be scalable with age (as opposed to being age-dependent systems). This can only be achieved by understanding how the articulation and corresponding acoustics develop with age. The different aspects of the project are therefore synergistic: findings from articulation and acoustic experiments will inform the development of algorithms essential to automatic speech recognition. Production data will include real-time 3D ultrasound recordings of the tongue, video recordings of the lips, palate impressions, microphone recordings, and accelerometer recordings of neck skin vibrations which have been shown to be beneficial in automatic speech and speaker recognition applications. The causal relationship between articulatory and acoustic variability will be explored, as will their relationship to misrecognition of child speech. Articulatory features will be incorporated into new automatic speech recognition systems along with acoustic features. The exploratory project will contribute to knowledge of variability between children, as well as variability over time as children grow and will provide, for the first time, normative data and scientific models. These can lead to robust child speech recognition systems as well as tools that will be useful for a variety of applications such as educational games, training of speech-language pathologists, automatic or semi-automatic transcription systems, and speech articulation visualization systems. It will train undergraduate and graduate students in important cross-disciplinary activities of technological and scientific significance. We believe that the proposed project is transformative in its advancement of the scientific and technological state of the art related to child speech.