IERI: Integrating Speech and User Modeling in a Reading Tutor that Listens

交互式系统如何通过倾听他们的用户来更好地建模？相反，系统如何通过利用用户模型来更好地监听？我们建议在一个跨学科挑战问题的背景下研究这些问题：开发一个儿童口语阅读的计算学生模型--即一个可以指导导师的学生个体知识、行为和学习的模型。这项工作将整合和扩展来自语音技术、认知心理学、用户建模和智能辅导的方法。拟议的工作将借鉴、贡献、测试和完善创新的、由技术支持的干预和独特研究平台的可扩展性：在计算机屏幕上显示故事的自动阅读辅导，使用语音识别器聆听儿童朗读，通过口语和图形辅助进行回应，并帮助他们学习阅读。它的听力能力使学生能够对阅读进度进行新颖的持续评估。截至2003年，有数百名儿童每天在9所学校的216台电脑上使用阅读辅导软件。拟议的工作将使用阅读导师捕获的语音和其他有价值的数据。使用阅读家教的孩子在阅读理解和其他技能方面的进步明显高于统计匹配的对照组。然而，它目前的有效性受到限制，因为它不能准确地倾听和建模学生。挑战是分析儿童的口语阅读，并在足够细微的粒度下评估各种组成部分的识字技能，以指导智能导师的决策，以适应学生的个人或集体教育需求。建立学生口语阅读模型的挑战将为指导工作和量化其成功提供问题驱动的基础。在阅读辅导中加入更好的模型，既可以测试它们的准确性，又可以立即将它们用于改进识字辅导。对照研究将调查连续版本的阅读辅导在多大程度上增加了不同类型学生的学习收益。阅读辅导中嵌入的自动化实验将帮助分析哪些辅导行为在哪些情况下有所帮助。拟议中的研究结合了广泛的技术和直接的社会影响。这项工作的预期技术贡献包括语音识别(特别是年轻和不流利的人)、用户建模、智能辅导以及理解和其他识字技能的自动评估方面的进展。更广泛的影响包括在研究期间使用阅读辅导的数千名儿童的识字能力提高，以及之后更多的儿童识字能力提高。

How can interactive systems model their users better by listening to them? Conversely, how can systems listen better by exploiting user models? We propose to investigate these questions in the context of a inter-disciplinary challenge problem: developing a computational student model of children's oral reading -- that is, a model of individual student knowledge, behavior, and learning that can guide a tutor. This work will integrate and extend methods from speech technology, cognitive psychology, user modeling, and intelligent tutors.The proposed work will draw on, contribute to, and test and refine the scalability of an innovative, technology-enabled intervention and unique research platform developed with previous NSF support: an automated Reading Tutor that displays stories on a computer screen, uses a speech recognizer to listen to children read aloud, responds with spoken and graphical assistance, and helps them learn to read. Its ability to listen enables novel continuous assessments of students' reading progress. As of 2003, the Reading Tutor was used daily by hundreds of children on 216 computers in nine schools. The proposed work will use speech and other valuable data captured by the Reading Tutor. Children who use the Reading Tutor have improved significantly more in reading comprehension and other skills than statistically matched controls. However, its current effectiveness is limited by inability to accurately hear and model the student.The challenge is to analyze children's oral reading and estimate various component literacy skills at a sufficiently fine grain size to guide the decisions of an intelligent tutor, so as to adapt to students' individual or collective educational needs.The challenge of developing a student model of oral reading will provide a problem-driven basis for guiding the work and quantifying its success. Incorporating successively better models in the Reading Tutor will both test their accuracy and put them to immediate use in improved literacy tutoring. Controlled studies will investigate how much successive versions of the Reading Tutor increase learning gains for different types of students. Automated experiments embedded in the Reading Tutor will help analyze which tutorial actions help in which cases.The proposed research combines broad technical and direct societal impact. Expected technical contributions of this work include advances in speech recognition (especially for young and disfluent speakers), user modeling, intelligent tutoring, and automated assessment of comprehension and other literacy skills. Broader impacts include improved literacy for thousands of children who use the Reading Tutor during the study, and many more thereafter.