Collaborative Research: Using Artificial Intelligence to Transform Online Video Lectures into Effective and Inclusive Agent-Based Presentations

协作研究：利用人工智能将在线视频讲座转变为有效且包容的基于代理的演示

• 批准号: 2201019
• 负责人: Nicoletta Adamo-Villani
• 金额: $ 54.67万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2201019&HistoricalAwards=false
• 关键词: Collaborative; Research; Using; Artificial; Intelligence

Students in introductory STEM courses frequently encounter video lectures that include an instructor standing next to a progression of slides. A challenge with these video lectures is that students may lose interest in science when they see the instructors as unhelpful (e.g., when they portray negative emotions while teaching) or unwelcoming (e.g., when they do not reflect the gender, ethnic, and racial diversity of the audience). This project aims to develop, validate, study, and make publicly available an artificial-intelligence (AI)-based framework that takes existing online instructional video lectures on introductory STEM topics created by human instructors and transforms them into instructionally effective and inclusive agent-based presentations. This work is intended to help improve science instruction and attract students from under-represented groups.The research team will apply artificial intelligence methods to extract gesture and voice from instructional videos with human instructors and transform them into instruction delivered by a diverse set of emotionally and socially sensitive onscreen animated embodied agents. Experimental research studies will investigate: 1) whether students learn better from an AI-transformed version of a video lecture than the original instructor-made video lecture; and 2) which features of onscreen agents in AI-transformed video lectures produce improved learning outcomes and processes, comparing delivery from an onscreen agent who does or does not match the student's gender, ethnicity, or race. Overall, results from the project will impact access to high-quality inclusive STEM learning experiences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在STEM入门课程中，学生经常会遇到视频讲座，其中包括一位教师站在幻灯片旁边。这些视频讲座的一个挑战是，当学生认为讲师毫无帮助时，他们可能会对科学失去兴趣（例如，当他们在教学时描绘负面情绪时）或不受欢迎（例如，当它们不反映观众的性别、民族和种族多样性时）。该项目旨在开发、验证、研究并公开一个基于人工智能（AI）的框架，该框架将现有的在线教学视频讲座转化为有效的、包容性的、基于代理的演示文稿。这项工作旨在帮助改善科学教学，吸引来自代表性不足群体的学生。研究团队将应用人工智能方法从人类教师的教学视频中提取手势和语音，并将其转换为由一组情感和社会敏感的动画化身代理提供的指令。实验研究将调查：1）学生是否从人工智能转换版本的视频讲座中学习得比原始的人工智能制作的视频讲座更好; 2）人工智能转换的视频讲座中的人工智能代理的哪些功能会产生更好的学习成果和过程，比较与学生性别，种族或种族匹配或不匹配的人工智能代理的交付。总体而言，该项目的成果将影响获得高质量的包容性STEM学习经验。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

项目成果

Animated Pedagogical Agents Performing Affective Gestures Extracted from the GEMEP Dataset: Can People Recognize Their Emotions?

动画教学代理执行从 GEMEP 数据集中提取的情感手势：人们能识别自己的情绪吗？

• 发表时间: 2024
• 期刊: Social Informatics and Telecommunications Engineering
• 作者: Magzhan Mukanova, Nicoletta Adamo

Recognizing and Relating to the Race/Ethnicity and Gender of Animated Pedagogical Agents

认识并联系动画教学人员的种族/民族和性别

• DOI: 10.1177/07356331231213932
• 发表时间: 2023
• 期刊: Journal of Educational Computing Research
• 影响因子: 4.8
• 作者: Zhao, Fangzheng;Mayer, Richard E.;Adamo-Villani, Nicoletta;Mousas, Christos;Choi, Minsoo;Lam, Luchcha;Mukanova, Magzhan;Hauser, Klay
• 通讯作者: Hauser, Klay

Developing a Scale for Measuring the Believability of Virtual Agents

开发衡量虚拟代理可信度的量表

• 发表时间: 2023
• 期刊: International Conference on Artificial Reality and Telexistence & Eurographics Symposium on Virtual Environments (ICAT-EGVE
• 作者: Guo, S.

Effects of Body Type and Voice Pitch on Perceived Audio-Visual Correspondence and Believability of Virtual Characters

体型和音高对虚拟角色感知视听对应性和可信度的影响

• DOI: 10.1145/3605495.3605791
• 发表时间: 2023
• 期刊: ACM Symposium on Applied Perception
• 作者: Lam, Luchcha;Choi, Minsoo;Mukanova, Magzhan;Hauser, Klay;Zhao, Fangzheng;Mayer, Richard;Mousas, Christos;Adamo-Villani, Nicoletta
• 通讯作者: Adamo-Villani, Nicoletta