Collaborative Research: Using Artificial Intelligence to Transform Online Video Lectures into Effective and Inclusive Agent-Based Presentations

STEM入门课程的学生经常会遇到视频讲座，其中包括一名教师站在一系列幻灯片旁边。这些视频讲座的一个挑战是，当学生认为讲师没有帮助(例如，当他们在教学中表现出负面情绪时)或不受欢迎(例如，当他们没有反映观众的性别、民族和种族多样性时)，他们可能会对科学失去兴趣。该项目旨在开发、验证、研究并公开提供一个基于人工智能(AI)的框架，该框架将由人类讲师创建的关于介绍性STEM主题的现有在线教学视频讲座转化为具有教学有效性和包容性的基于代理的演示。这项工作旨在帮助改善科学教学，并吸引来自代表性不足群体的学生。研究团队将应用人工智能方法，从有人类教师的教学视频中提取手势和语音，并将其转化为由一组不同的情感和社交敏感的屏幕动画具体化代理提供的教学。实验研究将调查：1)学生从人工智能转换版本的视频讲座中是否比最初的教师制作的视频讲座学习得更好；以及2)在人工智能转换的视频讲座中，屏幕代理的哪些功能可以改善学习结果和过程，并比较与学生的性别、种族或种族匹配或不匹配的屏幕代理的交付。总体而言，该项目的结果将影响获得高质量包容性STEM学习体验的机会。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Students in introductory STEM courses frequently encounter video lectures that include an instructor standing next to a progression of slides. A challenge with these video lectures is that students may lose interest in science when they see the instructors as unhelpful (e.g., when they portray negative emotions while teaching) or unwelcoming (e.g., when they do not reflect the gender, ethnic, and racial diversity of the audience). This project aims to develop, validate, study, and make publicly available an artificial-intelligence (AI)-based framework that takes existing online instructional video lectures on introductory STEM topics created by human instructors and transforms them into instructionally effective and inclusive agent-based presentations. This work is intended to help improve science instruction and attract students from under-represented groups.The research team will apply artificial intelligence methods to extract gesture and voice from instructional videos with human instructors and transform them into instruction delivered by a diverse set of emotionally and socially sensitive onscreen animated embodied agents. Experimental research studies will investigate: 1) whether students learn better from an AI-transformed version of a video lecture than the original instructor-made video lecture; and 2) which features of onscreen agents in AI-transformed video lectures produce improved learning outcomes and processes, comparing delivery from an onscreen agent who does or does not match the student's gender, ethnicity, or race. Overall, results from the project will impact access to high-quality inclusive STEM learning experiences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.