Collaborative Research: Using Artificial Intelligence to Transform Online Video Lectures into Effective and Inclusive Agent-Based Presentations

STEM入门课程的学生经常会遇到视频讲座，其中包括一位讲师站在一系列幻灯片旁边。这些视频讲座面临的一个挑战是，当学生认为教师不提供帮助（例如，当他们在教学中描绘负面情绪时）或不受欢迎（例如，当他们没有反映观众的性别、民族和种族多样性时）时，他们可能会失去对科学的兴趣。该项目旨在开发、验证、研究并公开一个基于人工智能（AI）的框架，该框架采用现有的由人类教师创建的介绍性STEM主题的在线教学视频讲座，并将其转化为具有教学效果和包容性的基于代理的演示文稿。这项工作旨在帮助改善科学教学，并吸引来自代表性不足群体的学生。研究小组将应用人工智能方法，从有人类教师的教学视频中提取手势和声音，并将其转换为由各种情感和社会敏感的屏幕动画实体代理提供的教学。实验研究将调查：1)学生从人工智能转换版本的视频讲座中是否比原始的教师制作的视频讲座学得更好；2)在人工智能转换的视频讲座中，屏幕代理的哪些特征产生了改进的学习成果和过程，比较了与学生的性别、种族或种族匹配或不匹配的屏幕代理的交付。总体而言，该项目的成果将影响获得高质量包容性STEM学习体验的机会。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Students in introductory STEM courses frequently encounter video lectures that include an instructor standing next to a progression of slides. A challenge with these video lectures is that students may lose interest in science when they see the instructors as unhelpful (e.g., when they portray negative emotions while teaching) or unwelcoming (e.g., when they do not reflect the gender, ethnic, and racial diversity of the audience). This project aims to develop, validate, study, and make publicly available an artificial-intelligence (AI)-based framework that takes existing online instructional video lectures on introductory STEM topics created by human instructors and transforms them into instructionally effective and inclusive agent-based presentations. This work is intended to help improve science instruction and attract students from under-represented groups.The research team will apply artificial intelligence methods to extract gesture and voice from instructional videos with human instructors and transform them into instruction delivered by a diverse set of emotionally and socially sensitive onscreen animated embodied agents. Experimental research studies will investigate: 1) whether students learn better from an AI-transformed version of a video lecture than the original instructor-made video lecture; and 2) which features of onscreen agents in AI-transformed video lectures produce improved learning outcomes and processes, comparing delivery from an onscreen agent who does or does not match the student's gender, ethnicity, or race. Overall, results from the project will impact access to high-quality inclusive STEM learning experiences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.