ITR: Creating the Next Generation of Intelligent Animated Conversational Agents

这是一个为期五年的连续奖励的第一年资助。本计划的目的是透过设计以电脑为基础的互动式阅读导师，结合新的语音和语言技术，以提高有阅读问题的儿童的阅读成就。 阅读辅导员将帮助讲英语和西班牙语的儿童学习阅读，向课堂教师和阅读专家提供工具，指导和练习阅读所需的一套听觉、视觉和语言技能、言语辨别、言语产生、语音意识、声音到字母的映射、词汇、流畅性和理解。 这些辅导老师将与阅读专家和教学设计师合作进行设计、测试和完善，并在科罗拉多博尔德小学的特殊教育项目中对儿童进行测试。 导师将采用新的和改进的听觉和视觉语音识别和面部动画技术。 五个合作伙伴站点--俄勒冈州研究生院（OGI）、普埃布拉美洲大学（UDLA）、加州大学圣克鲁斯分校（UCSC）、加州大学圣地亚哥分校（UCSD）和科罗拉多大学（CU）将开发语音和语言技术。 儿童语音识别器的研究和开发将在UDLA进行西班牙语和OGI进行英语。此外，这些网站将设计和开发语音语料库，以实现识别研究。 加州大学圣地亚哥分校将进行研究，导致头部跟踪和语音阅读系统的发展，并设计和开发视频语料库，使这项研究。 UCSC将进行研究，以开发具有改进动画功能的新动画面孔。系统集成将在OGI进行，OGI将把听觉和视觉识别系统以及面部动画系统集成到CSLU工具包中。CU将与教师、教学设计者和学生合作开发英语阅读导师，并对项目成果进行评估。UDLA还将开发和测试西班牙语版的导师，该项目预计将在听觉和视觉识别技术方面产生重大进展，包括准确识别儿童语音，准确识别语音的视觉特征，以及首次在语言培训应用中实时整合听觉和视觉语音识别。 此外，PI和他的团队将达到一个新的水平的理解儿童的语言结构，以及在阅读的听觉和视觉信息的处理。 面部动画预计将在吸引儿童方面发挥重要作用，使他们能够更多地享受学习体验，从而在任务上花费更多时间。 PI希望证明使用可见发音器官的面部动画将改善语音识别和语音生成技能，改善语音意识并改善阅读。 通过将听觉和视觉语音识别和语音生成技术集成到动画代理中，并设计将这些代理纳入精心设计的阅读计划的阅读导师，PI希望提高学校的阅读成绩。 为了优化这一结果，PI正在与阅读专家密切合作，以纳入他们的经验和最佳实践;并制定形成性和总结性评估计划，以确保公平和准确地评估计划干预的结果。

This is the first year funding of a five-year continuing award. The goal of this project is to improve reading achievement of children with reading problems by designing computer-based interactive reading tutors that incorporate new speech and language technologies. The reading tutors will help English- and Spanish-speaking children learn to read by providing classroom teachers and reading specialists with tools to instruct and exercise the set of auditory, visual and linguistic skills needed to read, speech discrimination, speech production, phonological awareness, sound-to-letter mappings, vocabulary, fluency and comprehension. The tutors will be designed, tested and refined in collaboration with reading specialists and instructional designers, and tested with children in special education programs in elementary schools in Boulder Colorado. The tutors will incorporate new and improved auditory and visual speech recognition and facial animation technologies. Five partner sites - Oregon Graduate Institute (OGI), Universidad de las Americas, Puebla (UDLA), University of California, Santa Cruz (UCSC), University of California, San Diego (UCSD) and the University of Colorado (CU), will develop speech and language technologies. Research and development of children's speech recognizers will be conducted at UDLA for Spanish and at OGI for English. In addition, these sites will design and develop speech corpora to enable recognition research. UCSD will conduct research leading to development of head tracking and speech reading systems, and design and develop video corpora to enable this research. UCSC will conduct research leading to development of new animated faces with improved animation capabilities. System integration will be conducted at OGI, which will integrate auditory and visual recognition systems and facial animation systems into the CSLU Toolkit. CU will develop English reading tutors in collaboration with teachers, instructional designers and students, and conduct evaluations of project outcomes. UDLA will also develop and test Spanish versions of the tutors.The project is expected to produce significant advances in auditory and visual recognition technologies, including accurate recognition of children's speech, accurate recognition of visual features of speech, and the first real-time integration of auditory and visual speech recognition in language training applications. In addition, the PI and his team will achieve a new level of understanding of the structure of children's speech, and the processing of auditory and visual information in reading. Facial animation is expected to play a major role in engaging children, enabling them to enjoy the learning experience more and therefore spend more time on task. The PI expects to demonstrate that facial animation using visible articulators will improve speech discrimination and speech production skills, improved phonological awareness and improved reading. By integrating auditory and visual speech recognition and speech generation technologies into animated agents, and designing reading tutors that incorporate these agents in a well designed reading program, the PI hopes to improve reading achievement in schools. To optimize this outcome, the PI is working closely with reading specialists to incorporate their experience and best practices; and by developing formative and summative evaluation plans that assure fair and accurate assessment of the outcomes of the planned interventions.