Collaborative Research: Multimodal Affective Pedagogical Agents for Different Types of Learners

虽然大多数关于具体化教学代理(指导和指导学习者的适应性虚拟代理)的研究都在探索认知特征，但本项目调查了代理情感/情感的作用。这个项目考察了在学习在线统计课程时，代理人的情感状态(例如，看起来感兴趣或关心)如何影响不同类型的学生(例如，不同的知识水平、性别、未被充分代表的群体地位、对STEM领域的兴趣以及个性特征)。该项目综合了几个领域的研究：a)计算机图形学研究在具身代理中逼真和可信的情感表示，b)人工智能和计算机视觉用于实时识别情感的先进方法和技术，c)从情感代理学习的认知心理学研究，以及d)情感代理对改善学生学习STEM概念的有效性的教育研究。通过实验研究，该项目将通过将有效的情绪调节的发现与支持具体化代理中逼真的情感表达的算法相结合，来促进代理设计和实现的最先进水平。为了研究情感教学代理的多通道设计特征，该项目有两个主要目标：(1)研究和开发新的情感识别算法和具体化动画代理中逼真的情感表征算法；(2)建立一个有经验的研究基础，指导不同类型学习者的情感教学代理设计。在一系列实验中，该项目将确定基于证据的设计原则，以指导展示情感/情感的代理的发展，包括哪种情感状态对哪种类型的学习者最有效。在第二系列实验中，该项目将实现一个网络摄像机系统来检测学习者的情绪状态(例如，困惑、感兴趣、内容或无聊)，调整代理在回应时显示的情绪状态。令人感兴趣的是，当教学主体对学习者的情绪状态敏感时，学生是否比不敏感时更好地学习统计学课程。除了它的科学价值，该项目还将开发和提供一个适应学习者特征的情感动画教学代理工具包，供所有年龄段的学习者使用，用于各种主题和环境的教育和培训。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

While most research on embodied pedagogical agents (adaptive virtual agents that guide and mentor learners) explores cognitive features, this project investigates the role of agent affect/emotion. This project examines how the agent's affective state (e.g., seeming interested or concerned) impacts different types of students (e.g., differing by knowledge level, gender, underrepresented group status, interest in STEM fields, and personality profile) when learning from online statistics lessons. The project integrates several areas of research: a) computer graphics research on life-like and believable representation of emotion in embodied agents, b) advanced methods and techniques from artificial intelligence and computer vision for real-time recognition of emotions, c) cognitive psychology research on learning from affective agents, and d) education research on the efficacy of affective agents for improving student learning of STEM concepts. Through experimental research the project will advance the state of the art in agent design and implementation by integrating findings on effective emotion regulation with algorithms that support life-like expression of emotions in embodied agents. To investigate the multimodal design features of affective pedagogical agents, the project has two main objectives: (1) research and develop novel algorithms for emotion recognition and for life-like emotion representation in embodied animated agents, and (2) develop an empirically grounded research base to guide the design of affective pedagogical agents for different types of learners. In one series of experiments the project will determine evidence-based design principles to guide the development of agents that demonstrate emotion/affect, including which kinds of affective states are most effective for which kinds of learners. In a second series of experiments, the project will implement a web-camera system to detect the emotional state of the learner (e.g., confused, interested, content, or bored), adapting the emotional state displayed by the agent in response. Of interest is whether students learn the statistics lesson better when the pedagogical agent is sensitive to the learner's emotional state than when it is not. In addition to its scientific merit, the project will develop and make available a toolkit of affective animated pedagogical agents that adapt to learner characteristics to be used by learners of all ages, for education and training in a variety of subject matters and settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.