IERI: Integrating Speech and User Modeling in a Reading Tutor that Listens

交互式系统如何通过倾听用户来更好地为他们建模？相反，系统如何通过利用用户模型来更好地倾听？我们建议在一个跨学科挑战问题的背景下研究这些问题：开发一个儿童口语阅读的计算学生模型——也就是说，一个可以指导导师的学生个体知识、行为和学习的模型。这项工作将整合和扩展语音技术、认知心理学、用户建模和智能导师的方法。拟议的工作将借鉴、贡献、测试和完善一个创新的、技术支持的干预和独特的研究平台的可扩展性，该平台是由以前的国家科学基金会支持开发的：一个自动阅读导师，它在计算机屏幕上显示故事，使用语音识别器听孩子大声朗读，用语音和图形辅助来回应，并帮助他们学习阅读。它的听力能力可以对学生的阅读进度进行新颖的持续评估。截至2003年，9所学校的216台电脑上每天有数百名儿童使用“阅读导师”。拟议的工作将使用语音和阅读导师捕获的其他有价值的数据。使用阅读导师的孩子在阅读理解和其他技能上的进步比统计上匹配的对照组明显更多。然而，它目前的有效性受到无法准确听到和模仿学生的限制。挑战在于分析儿童的口语阅读，并以足够细的粒度估计各种组成的读写技能，以指导智能导师的决策，从而适应学生的个人或集体教育需求。开发学生口头阅读模式的挑战将为指导工作和量化其成功提供一个问题驱动的基础。在阅读辅导中逐步纳入更好的模型将测试它们的准确性，并将它们立即用于改进的识字辅导。对照研究将调查阅读辅导的连续版本对不同类型学生的学习收益增加了多少。在阅读辅导中嵌入的自动实验将有助于分析哪些教程操作在哪些情况下有帮助。拟议的研究结合了广泛的技术和直接的社会影响。这项工作的预期技术贡献包括语音识别的进步（特别是对于年轻和不流利的说话者），用户建模，智能辅导，以及理解和其他读写技能的自动评估。更广泛的影响包括提高了数千名在研究期间使用阅读导师的儿童的读写能力，以及之后更多的儿童的读写能力。

How can interactive systems model their users better by listening to them? Conversely, how can systems listen better by exploiting user models? We propose to investigate these questions in the context of a inter-disciplinary challenge problem: developing a computational student model of children's oral reading -- that is, a model of individual student knowledge, behavior, and learning that can guide a tutor. This work will integrate and extend methods from speech technology, cognitive psychology, user modeling, and intelligent tutors.The proposed work will draw on, contribute to, and test and refine the scalability of an innovative, technology-enabled intervention and unique research platform developed with previous NSF support: an automated Reading Tutor that displays stories on a computer screen, uses a speech recognizer to listen to children read aloud, responds with spoken and graphical assistance, and helps them learn to read. Its ability to listen enables novel continuous assessments of students' reading progress. As of 2003, the Reading Tutor was used daily by hundreds of children on 216 computers in nine schools. The proposed work will use speech and other valuable data captured by the Reading Tutor. Children who use the Reading Tutor have improved significantly more in reading comprehension and other skills than statistically matched controls. However, its current effectiveness is limited by inability to accurately hear and model the student.The challenge is to analyze children's oral reading and estimate various component literacy skills at a sufficiently fine grain size to guide the decisions of an intelligent tutor, so as to adapt to students' individual or collective educational needs.The challenge of developing a student model of oral reading will provide a problem-driven basis for guiding the work and quantifying its success. Incorporating successively better models in the Reading Tutor will both test their accuracy and put them to immediate use in improved literacy tutoring. Controlled studies will investigate how much successive versions of the Reading Tutor increase learning gains for different types of students. Automated experiments embedded in the Reading Tutor will help analyze which tutorial actions help in which cases.The proposed research combines broad technical and direct societal impact. Expected technical contributions of this work include advances in speech recognition (especially for young and disfluent speakers), user modeling, intelligent tutoring, and automated assessment of comprehension and other literacy skills. Broader impacts include improved literacy for thousands of children who use the Reading Tutor during the study, and many more thereafter.