EXP: Partners in Learning: Building Rapport with a Virtual Peer Tutor

网络学习和未来学习技术计划资助的努力将有助于设想下一代学习技术，并推进我们对人们如何在技术丰富的环境中学习的了解。网络学习探索（EXP）项目通过设计和构建新型学习技术，研究其促进学习的可能性和有效使用它们的挑战，探索新型学习技术的可行性。该项目旨在了解和利用教师或导师如何通过构建支持融洽关系的技术与学习者建立融洽关系。研究将研究与学习者的融洽关系是什么样子，学生何时部署融洽关系技术，何时以及如何部署融洽关系技术（无论是通过人还是自动代理）增加学习，以及融洽关系如何随着时间的推移而演变。该项目将建立软件，可以帮助衡量学习者之间的融洽关系和计算机。该项目将开始通过建立一个多模态传感关系检测系统，基于计算机视觉，信号处理和机器学习的最新进展，将自动识别学习者与智能辅导系统交互期间的音频和视觉行为。人与人之间的辅导互动将用于指导融洽关系检测系统的开发。短期和纵向分析都将使用与基于AI的数学导师合作的学生进行，重点是他们的视觉行为（头部凝视估计将用于测量面部动作单元和姿势，如头部摆动或摇晃，以及人类之间的相互凝视），言语行为（使用CoreNLP和其他软件来检测表示与融洽相关的社会结构（例如礼貌、友谊等）的言语话语），以及夹带行为（同步或非同步、发散和收敛）。然后，该项目将设计RAPT，即融洽的同伴导师，它包括融洽检测系统和一个智能教学代理，该代理解释了融洽和非融洽的持久社会状态。模型/模拟的接口将被用来测试的设计之前，建立完整的教学代理。试验将在9-11年级的教室与智能几何导师使用两个迭代设计为基础的研究工作。

The Cyberlearning and Future Learning Technologies Program funds efforts that will help envision the next generation of learning technologies and advance what we know about how people learn in technology-rich environments. Cyberlearning Exploration (EXP) Projects explore the viability of new kinds of learning technologies by designing and building new kinds of learning technologies and studying their possibilities for fostering learning and challenges to using them effectively. This project seeks to understand, and capitalize on, how teachers or tutors build rapport with learners by building technologies that support rapport. Research will study what rapport with learners looks like, when students deploy rapport techniques, when and how deploying rapport techniques (whether by people or automated agents) increases learning, and how rapport evolves over time. The project will build software that can help measure rapport between learners and with computers. The project will begin by building a multimodal sensing rapport-detection system , based on recent advances in computer vision, signal processing, and machine learning which will automatically recognize audio and visual behaviors during learner interaction with an intelligent tutoring system. Human-human tutoring interactions will be used to guide development of the rapport detection system. Both short term and longitudinal analyses will be conducted using students working with an AI-based math tutor, focused on their visual behaviors (head gaze estimation will be used to measure facial action units and gestures like head nods or shakes, and mutual gaze between humans), verbal behaviors (using CoreNLP and other software to detect verbal utterances that represent rapport-related social constructs such as politeness, friendship, etc.), and entrainment behaviors (synchrony or asynchrony, divergence and convergence). The project will then design RAPT, the Rapport-Aligned Peer Tutor, which encompasses both the rapport detection system and an intelligent pedagogical agent that accounts for the persistent social states of rapport and non-rapport. Mockups/simulations of the interface will be used to test the designs before the full pedagogical agent is built. Trials will be conducted in 9-11 grade classrooms working with an intelligent geometry tutor using a two-iteration design-based research study.