Collaborative Research: Using Artificial Intelligence to Transform Online Video Lectures into Effective and Inclusive Agent-Based Presentations

在STEM入门课程中，学生经常会遇到视频讲座，其中包括一位教师站在幻灯片旁边。这些视频讲座的一个挑战是，学生可能会失去对科学的兴趣，当他们看到教师是无用的（例如，当他们在教学时描绘负面情绪时）或不受欢迎（例如，当它们不反映观众的性别、民族和种族多样性时）。该项目旨在开发、验证、研究并公开一个基于人工智能（AI）的框架，该框架将现有的在线教学视频讲座转化为有效的、包容性的、基于代理的演示文稿。这项工作旨在帮助改善科学教学，吸引来自代表性不足群体的学生。研究团队将应用人工智能方法从人类教师的教学视频中提取手势和语音，并将其转换为由一组情感和社会敏感的动画化身代理提供的指令。实验研究将调查：1）学生是否从人工智能转换版本的视频讲座中学习得比原始的人工智能制作的视频讲座更好; 2）人工智能转换的视频讲座中的人工智能代理的哪些功能会产生更好的学习成果和过程，比较与学生性别，种族或种族匹配或不匹配的人工智能代理的交付。总体而言，该项目的成果将影响获得高质量的包容性STEM学习经验。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Students in introductory STEM courses frequently encounter video lectures that include an instructor standing next to a progression of slides. A challenge with these video lectures is that students may lose interest in science when they see the instructors as unhelpful (e.g., when they portray negative emotions while teaching) or unwelcoming (e.g., when they do not reflect the gender, ethnic, and racial diversity of the audience). This project aims to develop, validate, study, and make publicly available an artificial-intelligence (AI)-based framework that takes existing online instructional video lectures on introductory STEM topics created by human instructors and transforms them into instructionally effective and inclusive agent-based presentations. This work is intended to help improve science instruction and attract students from under-represented groups.The research team will apply artificial intelligence methods to extract gesture and voice from instructional videos with human instructors and transform them into instruction delivered by a diverse set of emotionally and socially sensitive onscreen animated embodied agents. Experimental research studies will investigate: 1) whether students learn better from an AI-transformed version of a video lecture than the original instructor-made video lecture; and 2) which features of onscreen agents in AI-transformed video lectures produce improved learning outcomes and processes, comparing delivery from an onscreen agent who does or does not match the student's gender, ethnicity, or race. Overall, results from the project will impact access to high-quality inclusive STEM learning experiences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.