NRI: INT: COLLAB: Development, Deployment and Evaluation of Personalized Learning Companion Robots for Early Literacy and Language Learning

NRI：INT：COLLAB：用于早期识字和语言学习的个性化学习伴侣机器人的开发、部署和评估

• 批准号: 1734380
• 负责人: Abeer Alwan
• 金额: $ 61.56万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1734380&HistoricalAwards=false
• 关键词: NRI; INT; COLLAB; Development; Deployment

This National Robotics Initiative project will develop, deploy and evaluate personalized companion robots to assist kindergarten-age children in learning language and vocabulary skills. The aim is to accelerate the impacts of social robots for early education in schools and at home. The four-year project will advance knowledge in three key areas: (1) automatic speech recognition models for young children; (2) multi-modal student assessment algorithms for early language and literacy skills; and (3) personalization of activities, content, and dialogic question generation to boost learning outcomes. The project will generate new insights for how to develop expressive, socially responsive robots that provide more effective, engaging, and empathetic educational experiences for young children. To evaluate the impacts of long-term interactions on educational outcomes, the project team will conduct a 4-month study with Kindergarten classrooms, as well as a 3-month at-home study. The project will engage teachers and parents to develop key guidelines for best practices for use of social robots in classroom and home settings, and participating undergraduate and graduate students will be trained in the multidisciplinary aspects of social robotics, speech recognition and understanding, human participation studies, interactive machine learning for automatic assessment and personalization tools, and early education research. This research and development project will be implemented in two phases: An initial phase consisting of short pilot deployments to train and continually iterate development of project technologies and systems, followed by longer term deployment of the robot to examine autonomous interactions with social robots in school and home educational settings. During the development stage of individual components (automatic reading and language assessment tools, automatic question-generation algorithm, automatic speech recognition and spoken language understanding system models, and activities with the autonomous social robot learning companion) the project team will collect and analyze data with practical and performance measures, and refine and iterate each component of the system being developed. After development of the individual components, the autonomous social robot storytelling companion will be developed through repeated iterations with children. In the final year of the project, two 4-month studies will be conducted in six Kindergarten classrooms with 15 to 20 students each. This project is expected to result in five key contributions: (1) Development of Automatic Speech Recognition and Spoken Language Understanding systems for young children's speech, (2) Multi-modal automatic assessment algorithms for Kindergarten age children's spoken language and early reading skills; (3) Automatic personalization algorithms for story content customization and dialogic question generation in the context of young children's verbal storytelling; (4) Development of a fully autonomous, collaborative, peer-like social robot system with effective educational activities; and (5) Long-term studies with deployed social robots in schools and homes spanning several months and demonstrating sustained engagement and positive learning outcomes.

这个国家机器人计划项目将开发，部署和评估个性化的伴侣机器人，以帮助10岁以下的儿童学习语言和词汇技能。其目的是加速社交机器人对学校和家庭早期教育的影响。这个为期四年的项目将在三个关键领域推进知识：（1）幼儿自动语音识别模型;（2）早期语言和识字技能的多模式学生评估算法;（3）活动，内容和对话问题生成的个性化，以提高学习成果。该项目将为如何开发具有表达力、社会反应能力的机器人提供新的见解，为幼儿提供更有效、更吸引人、更有同情心的教育体验。为了评估长期互动对教育成果的影响，项目团队将对幼儿园教室进行为期4个月的研究，以及为期3个月的家庭研究。该项目将邀请教师和家长为在课堂和家庭环境中使用社交机器人的最佳实践制定关键指南，参与的本科生和研究生将接受社交机器人，语音识别和理解，人类参与研究，自动评估和个性化工具的交互式机器学习以及早期教育研究等多学科方面的培训。该研究和开发项目将分两个阶段实施：第一阶段包括短期试点部署，以培训和持续开发项目技术和系统，然后是机器人的长期部署，以研究在学校和家庭教育环境中与社交机器人的自主互动。在各个组件（自动阅读和语言评估工具，自动问题生成算法，自动语音识别和口语理解系统模型，以及与自主社交机器人学习伴侣的活动）的开发阶段，项目团队将收集和分析数据，并进行实际和性能测量，并完善和优化正在开发的系统的每个组件。在开发出各个组件之后，自主社交机器人讲故事伴侣将通过与儿童的反复迭代来开发。在项目的最后一年，将在六个幼儿园教室进行两次为期四个月的研究，每个教室有15至20名学生。本研究的主要成果包括：（1）开发幼儿语音自动识别和口语理解系统;（2）开发幼儿口语和早期阅读技能的多模态自动评估算法;（三）幼儿言语语境中故事内容定制和对话问题生成的自动个性化算法讲故事;（4）开发一个完全自主、协作、类似同伴的社交机器人系统，并提供有效的教育活动;（5）在学校和家庭中部署社交机器人的长期研究，持续数月，并展示持续的参与和积极的学习成果。

项目成果

Bi-APC: Bidirectional Autoregressive Predictive Coding for Unsupervised Pre-Training and its Application to Children’s ASR

• DOI: 10.1109/icassp39728.2021.9414970
• 发表时间: 2021-02
• 期刊: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Ruchao Fan;Amber Afshan;A. Alwan

Early Literacy and Oral Language Ties: Extending the range of human-computer interface for early assessment

早期识字和口语联系：扩展人机界面的范围以进行早期评估

• 发表时间: 2020
• 期刊: AERA (American Educational Research Association
• 作者: Bailey, A.;Martin, A.;Pogossian, A.;Perez, M.;Yeung, G.;Alwan, A.;Afshan, A.
• 通讯作者: Afshan, A.

On the Difficulties of Automatic Speech Recognition for Kindergarten-Aged Children

• DOI: 10.21437/interspeech.2018-2297
• 发表时间: 2018-09
• 作者: Gary Yeung;A. Alwan

An Improved Single Step Non-autoregressive Transformer for Automatic Speech Recognition

• DOI: 10.21437/interspeech.2021-1955
• 发表时间: 2021-06
• 作者: Ruchao Fan;Wei Chu;Peng Chang;Jing Xiao;A. Alwan

Subglottal resonances of American English speaking children

美国英语儿童的声门下共振

• DOI: 10.1121/1.5082289
• 发表时间: 2018
• 期刊: The Journal of the Acoustical Society of America
• 作者: Yeung, Gary;Lulich, Steven M.;Guo, Jinxi;Sommers, Mitchell S.;Alwan, Abeer
• 通讯作者: Alwan, Abeer