Collaborative Research: Multimodal Affective Pedagogical Agents for Different Types of Learners

虽然大多数研究体现教学代理（自适应虚拟代理，指导和指导学习者）探索认知功能，本项目调查代理的影响/情绪的作用。这个项目研究了代理人的情感状态（例如，似乎感兴趣或关心）影响不同类型的学生（例如，不同的知识水平，性别，代表性不足的群体地位，在STEM领域的兴趣和个性特征）从在线统计课程学习时。该项目整合了几个研究领域：a）计算机图形学研究在具体代理中逼真和可信的情感表示，B）人工智能和计算机视觉的先进方法和技术用于实时识别情感，c）从情感代理学习的认知心理学研究，以及d）情感代理的有效性教育研究，以提高学生学习STEM概念。通过实验研究，该项目将推进代理设计和实现的最新技术，将有效的情绪调节的研究结果与支持体现代理中情绪的逼真表达的算法相结合。为了研究情感教学代理的多模态设计特征，该项目有两个主要目标：（1）研究和开发新的情感识别算法和体现动画代理中的逼真情感表示，以及（2）开发一个基于经验的研究基础，以指导不同类型学习者的情感教学代理的设计。在一系列实验中，该项目将确定基于证据的设计原则，以指导开发展示情感/影响的代理，包括哪种情感状态对哪种学习者最有效。在第二系列实验中，该项目将实现一个网络摄像头系统来检测学习者的情绪状态（例如，困惑、感兴趣、满足或无聊），调整代理在响应中显示的情绪状态。感兴趣的是，学生是否学习统计课更好的教学代理是敏感的学习者的情绪状态比当它不是。除了其科学价值，该项目将开发和提供一个工具包的情感动画教学代理人，适应学习者的特点，供所有年龄段的学习者使用，在各种主题和设置的教育和培训。这一奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

While most research on embodied pedagogical agents (adaptive virtual agents that guide and mentor learners) explores cognitive features, this project investigates the role of agent affect/emotion. This project examines how the agent's affective state (e.g., seeming interested or concerned) impacts different types of students (e.g., differing by knowledge level, gender, underrepresented group status, interest in STEM fields, and personality profile) when learning from online statistics lessons. The project integrates several areas of research: a) computer graphics research on life-like and believable representation of emotion in embodied agents, b) advanced methods and techniques from artificial intelligence and computer vision for real-time recognition of emotions, c) cognitive psychology research on learning from affective agents, and d) education research on the efficacy of affective agents for improving student learning of STEM concepts. Through experimental research the project will advance the state of the art in agent design and implementation by integrating findings on effective emotion regulation with algorithms that support life-like expression of emotions in embodied agents. To investigate the multimodal design features of affective pedagogical agents, the project has two main objectives: (1) research and develop novel algorithms for emotion recognition and for life-like emotion representation in embodied animated agents, and (2) develop an empirically grounded research base to guide the design of affective pedagogical agents for different types of learners. In one series of experiments the project will determine evidence-based design principles to guide the development of agents that demonstrate emotion/affect, including which kinds of affective states are most effective for which kinds of learners. In a second series of experiments, the project will implement a web-camera system to detect the emotional state of the learner (e.g., confused, interested, content, or bored), adapting the emotional state displayed by the agent in response. Of interest is whether students learn the statistics lesson better when the pedagogical agent is sensitive to the learner's emotional state than when it is not. In addition to its scientific merit, the project will develop and make available a toolkit of affective animated pedagogical agents that adapt to learner characteristics to be used by learners of all ages, for education and training in a variety of subject matters and settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.