Collaborative Research: Improving Speech Technology for Better Learning Outcomes: The Case of AAE Child Speakers

合作研究：改进语音技术以获得更好的学习成果：AAE 儿童演讲者的案例

• 批准号: 2202049
• 负责人: Mari Ostendorf
• 金额: $ 26.01万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202049&HistoricalAwards=false
• 关键词: Collaborative; Research; Improving; Speech; Technology

The lack of reading proficiency seen in children of underserved school districts has lasting impacts on students’ performances in various subjects. Low literacy is an especially pressing issue for African American students. Interactive spoken language systems offer the possibility of a powerful tool for assisting in early childhood education, freeing up teachers’ time, and engaging students in repeated opportunities for learning. These systems involve both Automatic Speech Recognition and Text-to-Speech Systems. The goal of this research is to improve the performance of such systems for young speakers of African American English (AAE) such that automated oral literacy assessment can be developed. The research has important societal and technological impacts. It will enhance the usability of speech technology in early education for AAE speaking children, providing a model for better supporting students with diverse dialects. Many under-resourced children do not have access to adequate reading and language assessments, and the proposed work will address these issues by creating methods for adapting spoken language technology to AAE children, increasing fairness in speech technology on a broader scale. The work has strong outreach and dissemination programs and will train undergraduate and graduate students in interdisciplinary research in Electrical and Computer Engineering, Linguistics, Education, and Psychology. Challenges facing children’s Automatic Speech Recognition (ASR) are due to (1) lack of child speech data and, hence, current models used for recognition are trained using data collected from adult speakers, and (2) children display a wider range of intra- and inter- speaker variability than adults. ASR performance is especially poor for children who are non-native English speakers or those who at times transition into dialects such as AAE that are different from what ASR systems are typically trained on. In addition, most dialog systems built on text-to-speech (TTS) technology are designed using General American English (GAE) voices, which minority children may not identify with. In the high-stakes area of education, these considerations impact the effectiveness of technology for different groups. The work will utilize a new and continuously developing database of AAE children's speech to research the impact of spoken language systems on children’s learning outcomes. On the learning side, the research will highlight the impact of dialect on literacy assessment. On the technology side, the work will yield novel machine learning algorithms for low-resource tasks. Specifically, this project will develop data augmentation techniques that can increase the amount of training data available for low-resource tasks, and data normalization techniques so that ASR performance is improved for AAE child speakers. The work on TTS will explore new methods of disentangling speaker and dialect impacts on spectral realization of phrases that model dialect density (rather than treating dialect as a categorical variable) and separately accounting for pronunciation and prosodic factors. Methods found to be effective for TTS will be leveraged in the data augmentation work for ASR and explored as a diagnostic in literacy assessment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在服务水平低下的学区，孩子们缺乏阅读能力，这对学生在各个学科的表现产生了持久的影响。识字率低是非洲裔美国学生面临的一个特别紧迫的问题。 交互式口语系统提供了一种强大的工具，可以帮助幼儿教育，腾出教师的时间，并让学生有机会重复学习。这些系统涉及自动语音识别和文本到语音系统。本研究的目标是提高这些系统的性能，为年轻的非洲裔美国人英语（AAE），这样的自动化口语识字评估可以开发。该研究具有重要的社会和技术影响。它将提高语音技术在AAE儿童早期教育中的可用性，为更好地支持具有不同方言的学生提供模式。许多资源不足的儿童无法获得充分的阅读和语言评估，拟议的工作将通过创建使口语技术适应AAE儿童的方法来解决这些问题，在更广泛的范围内增加语音技术的公平性。这项工作有很强的推广和传播计划，并将培养本科生和研究生在电气和计算机工程，语言学，教育和心理学的跨学科研究。儿童的自动语音识别（ASR）面临的挑战是由于（1）缺乏儿童语音数据，因此，用于识别的当前模型使用从成人说话者收集的数据进行训练，以及（2）儿童显示出比成人更宽范围的说话者内和说话者间变异性。 对于非英语母语的儿童或那些有时会过渡到方言（如AAE）的儿童来说，ASR的表现尤其糟糕，这些方言与ASR系统通常训练的方言不同。此外，大多数基于文本到语音（TTS）技术的对话系统都是使用通用美式英语（GAE）语音设计的，少数民族儿童可能无法识别。在教育的高风险领域，这些考虑因素影响到技术对不同群体的有效性。这项工作将利用一个新的和不断发展的AAE儿童的语音数据库，研究口语系统对儿童学习成果的影响。在学习方面，研究将突出方言对识字评估的影响。在技术方面，这项工作将为低资源任务产生新的机器学习算法。 具体而言，该项目将开发数据增强技术，可以增加低资源任务的训练数据量，以及数据规范化技术，以便提高AAE儿童扬声器的ASR性能。TTS的工作将探索新的方法来解开说话者和方言对短语频谱实现的影响，这些短语模型方言密度（而不是将方言视为分类变量）并分别考虑发音和韵律因素。 被发现对TTS有效的方法将在ASR的数据增强工作中得到利用，并作为识字评估的诊断进行探索。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。