EAGER: Collaborative Research: Models of Child Speech

与已经研究了几十年的成人语音的产生、建模和机器识别相比，儿童语音的产生、声学建模和识别还没有得到同样程度的关注。随着儿童言语和语言发展的新应用变得越来越重要和普遍，缺乏处理儿童言语的学术资源是有问题的。对于小学生来说尤其如此。随着孩子的成长，他们的发音器也在成长，从而导致他们说话的声音发生变化。例如，一个6岁的孩子说的“阳光”这个词的波形可能与同一个孩子9岁时的波形完全不同。这就是为什么目前机器对儿童语言的识别在幼儿中表现不佳，也不能随着孩子的成长而扩大。也就是说，这些系统往往与年龄有关。随着儿童的成长，理解和建模儿童语言不仅对发展更好的识别系统很重要，而且对更好地理解和诊断语言病理学（SLP）也很重要。随着儿童早期发育缺陷的长期负面影响得到越来越广泛的认识，针对幼儿的社会和保健服务及技术应用的机会和需求正在增加。特别是，现在人们已经认识到，语言发展和读写能力的早期缺陷会持续到成年，公立学校对特殊语言帮助服务的需求大大超过了供应。因此，临床医生和教师不再能够为每个孩子提供最有效的治疗或必要的关注。更好的语音识别系统将为改进诊断和更强的计算机治疗提供机会。这个探索性研究项目的早期资助旨在模拟语音和语言在小学期间是如何发展的，以及语言障碍儿童在语音发音方面的差异。儿童语言模型将导致计算机程序的发展，这些程序可以用于教育和治疗目的。从科学上讲，该探索性项目将1)通过独特的发音和声学分析相结合，揭示20-26岁小学适龄儿童的语音生产发展过程；2)开发声学模型，并最终开发可随年龄扩展的儿童语音自动语音识别系统（而不是年龄依赖系统）。这只能通过理解发音和相应的声学如何随着年龄的增长而发展来实现。因此，该项目的不同方面是协同的：来自发音和声学实验的发现将为自动语音识别必不可少的算法的开发提供信息。生产数据将包括舌头的实时3D超声记录、嘴唇的视频记录、上颚印象、麦克风记录和颈部皮肤振动的加速度计记录，这些记录已被证明在自动语音和说话人识别应用中是有益的。将探讨发音和声学变异之间的因果关系，以及它们与儿童言语误认的关系。发音特征将与声学特征一起纳入新的自动语音识别系统。探索性项目将有助于了解儿童之间的差异以及儿童成长过程中随时间变化的差异，并将首次提供规范性数据和科学模型。这些可以产生强大的儿童语音识别系统，以及对各种应用有用的工具，如教育游戏、语言病理学家的培训、自动或半自动转录系统以及语音发音可视化系统。它将培养本科生和研究生进行具有重要技术和科学意义的跨学科活动。我们认为，拟议的项目在推动与儿童语言有关的科学和技术状况方面具有变革性。

In contrast to the production, modeling, and machine recognition of adult speech, which have been studied for decades, the production, acoustic modeling and recognition of child speech have not received the same level of attention. The lack of scholarly resources for dealing with children's speech is problematic as new applications for child speech and language development become increasingly important and commonplace. This is especially true for elementary school children. As children grow, their articulators grow as well, resulting in variations in their speech sounds. For example, the waveform of the word 'sunny' spoken by a 6-year old can be quite different than that of the same child when she is 9 years old. This is why current machine recognition of children's speech does not perform well for young children and does not scale up as the child grows. That is, these systems tend to be age dependent. Understanding and modeling child speech as children grow is important not only to developing better recognition systems but also for better understanding and diagnosis of speech-language pathology (SLP). As the negative long-term ramifications of deficits in early childhood development gain increasingly broad recognition, the opportunities and the need for social and health services and technological applications targeted toward young children are growing. In particular, it is now understood that early deficits in language development and literacy persist into adulthood, and the demand for SLP services in public schools is significantly outpacing supply. As a result, it is no longer feasible for clinicians and teachers to provide the most effective treatments or the necessary attention to every child. Better speech recognition systems would provide an opportunity for improved diagnosis and more intense computer-based therapy. This Early Grant for Exploratory Research project aims to model how speech and language develop during elementary school and how children with speech disorders differ in their articulation of speech sounds. Models of child speech will lead to the development of computer programs which can be used for educational as well as therapeutic purposes.Scientifically, the exploratory project will 1) reveal processes of speech production development in 20-26 elementary school-aged children through a unique combination of articulatory and acoustic analyses, and 2) develop acoustic models and eventually automatic speech recognition systems for children's speech which can be scalable with age (as opposed to being age-dependent systems). This can only be achieved by understanding how the articulation and corresponding acoustics develop with age. The different aspects of the project are therefore synergistic: findings from articulation and acoustic experiments will inform the development of algorithms essential to automatic speech recognition. Production data will include real-time 3D ultrasound recordings of the tongue, video recordings of the lips, palate impressions, microphone recordings, and accelerometer recordings of neck skin vibrations which have been shown to be beneficial in automatic speech and speaker recognition applications. The causal relationship between articulatory and acoustic variability will be explored, as will their relationship to misrecognition of child speech. Articulatory features will be incorporated into new automatic speech recognition systems along with acoustic features. The exploratory project will contribute to knowledge of variability between children, as well as variability over time as children grow and will provide, for the first time, normative data and scientific models. These can lead to robust child speech recognition systems as well as tools that will be useful for a variety of applications such as educational games, training of speech-language pathologists, automatic or semi-automatic transcription systems, and speech articulation visualization systems. It will train undergraduate and graduate students in important cross-disciplinary activities of technological and scientific significance. We believe that the proposed project is transformative in its advancement of the scientific and technological state of the art related to child speech.