Collaborative Research: Multimodal Affective Pedagogical Agents for Different Types of Learners

虽然大多数关于具身教学代理（引导和指导学习者的自适应虚拟代理）的研究都在探索认知特征，但本项目研究了代理情感/情感的作用。该项目研究了代理在学习在线统计学课程时的情感状态（例如，似乎感兴趣或关心）如何影响不同类型的学生（例如，不同的知识水平、性别、未被充分代表的群体地位、对STEM领域的兴趣和个性特征）。该项目整合了几个研究领域：a)计算机图形学研究在具具代理中逼真和可信的情感表示，b)人工智能和计算机视觉的先进方法和技术用于实时识别情绪，c)情感代理学习的认知心理学研究，d)情感代理提高学生STEM概念学习效果的教育研究。通过实验研究，该项目将通过将有效情绪调节的发现与支持具身代理中逼真情绪表达的算法相结合，推进代理设计和实施的最新技术。为了研究情感教学代理的多模态设计特征，本项目有两个主要目标：(1)研究和开发在具身动画代理中进行情感识别和逼真情感表示的新算法；(2)建立基于经验的研究基础，指导针对不同类型学习者的情感教学代理的设计。在一系列实验中，该项目将确定基于证据的设计原则，以指导展示情感/情感的代理的开发，包括哪种情感状态对哪种学习者最有效。在第二个系列的实验中，该项目将实现一个网络摄像头系统来检测学习者的情绪状态（例如，困惑、感兴趣、满足或无聊），并调整智能体所显示的情绪状态作为回应。令人感兴趣的是，当教学主体对学习者的情绪状态敏感时，学生是否能更好地学习统计学课程。除了其科学价值外，该项目还将开发和提供一套适合学习者特点的情感动画教学代理工具，供所有年龄的学习者使用，用于各种主题和环境的教育和培训。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

While most research on embodied pedagogical agents (adaptive virtual agents that guide and mentor learners) explores cognitive features, this project investigates the role of agent affect/emotion. This project examines how the agent's affective state (e.g., seeming interested or concerned) impacts different types of students (e.g., differing by knowledge level, gender, underrepresented group status, interest in STEM fields, and personality profile) when learning from online statistics lessons. The project integrates several areas of research: a) computer graphics research on life-like and believable representation of emotion in embodied agents, b) advanced methods and techniques from artificial intelligence and computer vision for real-time recognition of emotions, c) cognitive psychology research on learning from affective agents, and d) education research on the efficacy of affective agents for improving student learning of STEM concepts. Through experimental research the project will advance the state of the art in agent design and implementation by integrating findings on effective emotion regulation with algorithms that support life-like expression of emotions in embodied agents. To investigate the multimodal design features of affective pedagogical agents, the project has two main objectives: (1) research and develop novel algorithms for emotion recognition and for life-like emotion representation in embodied animated agents, and (2) develop an empirically grounded research base to guide the design of affective pedagogical agents for different types of learners. In one series of experiments the project will determine evidence-based design principles to guide the development of agents that demonstrate emotion/affect, including which kinds of affective states are most effective for which kinds of learners. In a second series of experiments, the project will implement a web-camera system to detect the emotional state of the learner (e.g., confused, interested, content, or bored), adapting the emotional state displayed by the agent in response. Of interest is whether students learn the statistics lesson better when the pedagogical agent is sensitive to the learner's emotional state than when it is not. In addition to its scientific merit, the project will develop and make available a toolkit of affective animated pedagogical agents that adapt to learner characteristics to be used by learners of all ages, for education and training in a variety of subject matters and settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.