EXP: Partners in Learning: Building Rapport with a Virtual Peer Tutor

网络学习和未来学习技术计划资助的努力将有助于展望下一代学习技术，并促进我们对人们在技术丰富的环境中如何学习的了解。网络学习探索(EXP)项目通过设计和构建新的学习技术，并研究它们促进学习的可能性和有效使用它们的挑战，来探索新类型的学习技术的可行性。该项目旨在了解并利用教师或导师如何通过构建支持融洽的技术来与学生建立融洽的关系。研究将研究与学习者的融洽是什么样子，学生何时部署融洽的技术，何时以及如何使用融洽的技术(无论是通过人工还是自动代理)来增加学习，以及融洽是如何随着时间的推移而发展的。该项目将开发软件，帮助衡量学习者之间以及与计算机之间的融洽程度。该项目将首先建立一个多模式传感融洽检测系统，基于计算机视觉、信号处理和机器学习的最新进展，该系统将在学习者与智能教学系统交互期间自动识别音频和视觉行为。人与人之间的辅导互动将被用来指导融洽检测系统的开发。短期和纵向分析都将使用与基于人工智能的数学导师合作的学生进行，重点是他们的视觉行为(头部凝视估计将被用来衡量面部动作单位和手势，如点头或摇头，以及人与人之间的相互凝视)，言语行为(使用CoreNLP和其他软件来检测代表融洽相关社会结构的言语，如礼貌、友谊等)，以及夹带行为(同步或异步、分歧和收敛)。然后，该项目将设计RAPT，这是一个基于融洽的同伴导师，它包括融洽的检测系统和一个智能教学代理，该代理解释了融洽和非融洽的持久社会状态。在构建完整的教学代理之前，将使用界面的模型/模拟来测试设计。试验将在9-11年级的教室中进行，与智能几何教师合作，使用基于设计的两次迭代研究性学习。

The Cyberlearning and Future Learning Technologies Program funds efforts that will help envision the next generation of learning technologies and advance what we know about how people learn in technology-rich environments. Cyberlearning Exploration (EXP) Projects explore the viability of new kinds of learning technologies by designing and building new kinds of learning technologies and studying their possibilities for fostering learning and challenges to using them effectively. This project seeks to understand, and capitalize on, how teachers or tutors build rapport with learners by building technologies that support rapport. Research will study what rapport with learners looks like, when students deploy rapport techniques, when and how deploying rapport techniques (whether by people or automated agents) increases learning, and how rapport evolves over time. The project will build software that can help measure rapport between learners and with computers. The project will begin by building a multimodal sensing rapport-detection system , based on recent advances in computer vision, signal processing, and machine learning which will automatically recognize audio and visual behaviors during learner interaction with an intelligent tutoring system. Human-human tutoring interactions will be used to guide development of the rapport detection system. Both short term and longitudinal analyses will be conducted using students working with an AI-based math tutor, focused on their visual behaviors (head gaze estimation will be used to measure facial action units and gestures like head nods or shakes, and mutual gaze between humans), verbal behaviors (using CoreNLP and other software to detect verbal utterances that represent rapport-related social constructs such as politeness, friendship, etc.), and entrainment behaviors (synchrony or asynchrony, divergence and convergence). The project will then design RAPT, the Rapport-Aligned Peer Tutor, which encompasses both the rapport detection system and an intelligent pedagogical agent that accounts for the persistent social states of rapport and non-rapport. Mockups/simulations of the interface will be used to test the designs before the full pedagogical agent is built. Trials will be conducted in 9-11 grade classrooms working with an intelligent geometry tutor using a two-iteration design-based research study.