RI: Small: Early Elementary Reading Verification in Challenging Acoustic Environments

RI：小：具有挑战性的声学环境中的早期小学阅读验证

• 批准号: 2008043
• 负责人: Eric Fosler-Lussier
• 金额: $ 45万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008043&HistoricalAwards=false
• 关键词: RI; Small; Early; Elementary; Reading

Young students who have difficulty in reading fluently can gain confidence by repeatedly reading story passages. Computer reading assessment can help improve student’s fluency while freeing up teacher to help students more efficiently. In previous projects, computer-based technologies have been shown to improve the reading ability of low performing readers in urban schools. Reading verification is a critical task, determining whether a student has correctly read the words on the page, or has mispronounced words, is stuck, or has jumped ahead or back. However, current systems that use automatic speech recognition (ASR) tend towards accepting incorrect pronunciations, instead of figuring out mistakes, and operate poorly when there is noise. This project addresses these issues by studying how to build a reliable computer system that can tell, in real-time, whether students are reading correctly within a noisy classroom, and make it available on many different types of computers. To allow other researchers to compare their studies, the project team will create and make available a dataset of reading examples recorded in noise through the Language Research Lab at the Center of Science and Industry (COSI) museum. By interacting with lab personnel, visitors will be educated on how speech recognition systems work as well as on how language science research is done. Thus, if this project is successful, significantly better computer-based reading assessment technologies will be developed, which will help many children read better, and the public will become better informed about reading, speech technologies and science in general. The approach investigated in this project abandons the need for full ASR technology, instead performing Reading Verification (RV) directly on the speech signal while jointly performing speech enhancement to focus on the child's speech. The elimination of ASR decoding technology permits creation of smaller verification models that can be deployed on devices or web browsers. Building on pilot work in mispronunciation detection and integration of phonetic information into speech enhancement, a new attention-based tracking approach is examined that directly models phenomena like partial word disfluencies, explores the difference between enhancement for child and adult speech, and investigates a novel joint model of detection and enhancement integrating longitudinal adaptation techniques. The research activities are supported by a new collection of speech data, consisting of stories read in a noisy science museum environment. The recorded dataset will facilitate comparative work in reading assessment and will be released as part of the Ohio Child Speech Corpus.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

阅读有困难的青年学生可以通过反复阅读故事短文来获得信心。计算机阅读测评可以帮助提高学生的流利性，同时也解放了教师更有效地帮助学生。在以前的项目中，基于计算机的技术已经被证明可以提高城市学校表现不佳的读者的阅读能力。阅读验证是一项关键任务，它确定学生是否正确阅读了页面上的单词，或者读错了单词，被卡住了，或者跳到了前面或后面。然而，目前使用自动语音识别(ASR)的系统倾向于接受错误的发音，而不是找出错误，并且在有噪音时运行不佳。这个项目通过研究如何建立一个可靠的计算机系统来解决这些问题，该系统可以在嘈杂的教室中实时判断学生是否正确阅读，并使其在许多不同类型的计算机上可用。为了让其他研究人员比较他们的研究，该项目团队将通过科学与工业中心(COSI)博物馆的语言研究实验室创建并提供一个在噪音中记录的阅读实例数据集。通过与实验室人员的互动，参观者将学习语音识别系统如何工作，以及语言科学研究是如何进行的。因此，如果该项目成功，将开发出更好的基于计算机的阅读评估技术，这将帮助许多儿童更好地阅读，公众将更好地了解阅读、语音技术和一般科学。本项目研究的方法不需要完全的ASR技术，而是直接对语音信号进行阅读验证(RV)，同时联合执行语音增强以专注于儿童的语音。ASR解码技术的取消允许创建更小的验证模型，这些模型可以部署在设备或Web浏览器上。在错误发音检测和语音信息集成到语音增强的试点工作的基础上，研究了一种新的基于注意力的跟踪方法，该方法直接对部分单词不流利等现象进行建模，探索了儿童和成人语音增强的差异，并结合纵向适应技术研究了一种新的检测和增强的联合模型。研究活动得到了一组新的语音数据的支持，这些数据包括在嘈杂的科学博物馆环境中阅读的故事。记录的数据集将促进阅读评估的比较工作，并将作为俄亥俄州儿童言语语料库的一部分发布。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Fine-Grained Textual Knowledge Transfer to Improve RNN Transducers for Speech Recognition and Understanding

细粒度文本知识转移可改进 RNN 转换器以实现语音识别和理解

• DOI: 10.1109/icassp49357.2023.10094997
• 发表时间: 2023
• 期刊: Speech and Signal Processing
• 作者: Sunder, Vishal;Thomas, Samuel;Kuo, Hong-Kwang J.;Kingsbury, Brian;Fosler-Lussier, Eric
• 通讯作者: Fosler-Lussier, Eric

End-to-End Word-Level Disfluency Detection and Classification in Children’s Reading Assessment

• DOI: 10.1109/icassp49357.2023.10095555
• 发表时间: 2023-06
• 期刊: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Lavanya Venkatasubramaniam;Vishal Sunder;E. Fosler-Lussier