Collaborative Research: Student Affect Detection and Intervention with Teachers in the Loop

近年来，人们越来越多地将现代人工智能技术整合到自适应学习系统中，以提高学生的学习能力。一个关键的新兴领域是在使用模型，可以识别学生的情绪在上下文中，被称为情感状态。这些模型通常采用机器学习分类器的形式，识别学生与在线学习系统交互的影响。在这个项目中，研究人员将开发自适应学习系统，积极争取教师的帮助，以开发更好的学生情感检测方法。作为回报，该系统将支持教师的工作，为他们提供关于每个学生情感状态的实时报告。然后，该系统将学习模仿教师为不参与的学生选择干预方法，以便自动提供干预。总的来说，该项目预计将导致i）更好地理解如何利用和调整教师的观点来检测和应对影响，ii）加强教师和自动化软件的干预，重新吸引学生并提高学习成果。该项目将分为三个阶段。首先，研究人员将采用主动机器学习方法，要求教师在课堂活动中休息时观察特定的学生;这些方法可以通过提供关于情感状态信息量最大的学生的数据来提高情感检测器的质量，以改进分类器，而不是通过以循环方式观察学生来开发这些检测器的标准方法。其次，研究人员将把更丰富的数据类型（特别是自我报告的情感标签的置信度）纳入检测器中，以提高其质量。这些自我报告的信心评级反映了人类对特定影响判断的不确定性，这将与分类器的不确定性进行比较，以可能揭示对学生影响的见解，例如影响模糊的情况下的属性是什么。第三，研究人员将使用众包来征求教师的意见，以确定特定的情感干预何时适合特定的学生，并将使用强化学习开发自动化干预方法。这些自动干预方法具有高度可扩展性，因为它们可以使系统能够采取教师将采取的干预行动，以支持同时经历负面影响的不同学生。这一干预系统将在真实的课堂上进行测试，学生将在ASSISTments学习，ASSISTments是一个免费的网络学习平台，每年有60 000多名学生使用。如果成功，这个项目将导致新的科学发现的动态影响和新技术的可扩展的学生影响检测和干预。这个奖项反映了NSF的法定使命，并已被认为是值得通过评估使用基金会的智力价值和更广泛的影响审查标准的支持。

In recent years, there has been increasing effort to integrate modern artificial intelligence technologies into adaptive learning systems to enhance student learning. One key emerging area is in the use of models that can recognize student emotion in context, referred to as affective states. These models typically take the form of machine learning classifiers that recognize affect from the student's interaction with an online learning system. In this project, the investigators will develop adaptive learning systems that actively enlist the help of teachers to develop better student affect detection methods. In return, the system will support the work of teachers by providing them reports on the affective state of each student in real-time. The system will then learn to mimic teachers' choices of intervention methods for disengaged students in order to deliver interventions automatically. Overall, this project is anticipated to lead to i) better understanding of how to leverage and align to teachers' perspectives in detecting and responding to affect, and ii) enhanced intervention by both teachers and automated software that re-engages students and improves learning outcomes.This project will be organized into three phases. First, the investigators will employ active machine learning methods to ask teachers to observe specific students when they have a break in classroom activity; these methods can improve the quality of the affect detectors by providing data on the students whose affective states are most informative to improve the classifier, rather than the standard method of developing these detectors by observing students in round-robin fashion. Second, the investigators will incorporate richer data types (specifically, self-reported confidence ratings of affect labels) into the detectors to improve their quality. These self-reported confidence ratings reflect how uncertain humans are about specific affect judgements, which will be compared to the uncertainty of classifiers, to possibly reveal insights into student affect, such as what the properties are of situations where affect is ambiguous. Third, the investigators will use crowdsourcing to solicit ideas from teachers as to when specific affect interventions will be appropriate for specific students, and will develop automated intervention methods using reinforcement learning. These automated intervention methods are highly scalable since they can enable the system to take the actions the teacher would take to intervene to support different students experiencing negative affect at the same time. This intervention system will be tested in real classrooms as students learn within ASSISTments, a free web-based learning platform used by over 60,000 students a year. If successful, this project will lead to new scientific discoveries on the dynamics of affect and new technology for scalable student affect detection and intervention.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.