Collaborative Research: Student Affect Detection and Intervention with Teachers in the Loop

近年来，越来越多的人努力将现代人工智能技术整合到自适应学习系统中，以加强学生的学习。一个关键的新兴领域是使用能够在情境中识别学生情绪的模型，称为情感状态。这些模型通常采用机器学习分类器的形式，识别学生与在线学习系统交互的影响。在这个项目中，研究人员将开发自适应学习系统，积极争取教师的帮助，以开发更好的学生情感检测方法。作为回报，该系统将通过向教师提供关于每个学生的情感状态的实时报告来支持他们的工作。然后，该系统将学习模仿教师为未参与的学生选择的干预方法，以便自动提供干预。总体而言，这一项目预计将导致：1)更好地理解如何在检测和应对情感方面利用并与教师的观点保持一致；2)加强教师和自动化软件的干预，使学生重新参与进来，并改善学习结果。首先，调查人员将使用主动机器学习方法，要求教师观察特定学生在课堂活动中的休息时间；这些方法可以通过提供情感状态最具信息量的学生的数据来改进分类器，而不是通过循环观察学生来开发这些检测器的标准方法，从而提高情感检测器的质量。其次，调查人员将把更丰富的数据类型(具体地说，影响标签的自我报告置信度评级)纳入检测器，以提高其质量。这些自我报告的信心评分反映了人类对具体情感判断的不确定性，这将与分类器的不确定性进行比较，以可能揭示对学生情感的洞察，例如情感模棱两可的情景的属性是什么。第三，调查人员将使用众包来征求教师的意见，以确定何时特定的情感干预适合特定的学生，并将利用强化学习开发自动干预方法。这些自动干预方法具有高度的可扩展性，因为它们可以使系统能够采取教师将采取的干预措施，以支持同时经历负面影响的不同学生。这个干预系统将在真实的教室中进行测试，学生在ASSISTments学习，ASSISTments是一个免费的基于网络的学习平台，每年有超过6万名学生使用。如果成功，该项目将在情感动态方面带来新的科学发现，并为可扩展的学生情感检测和干预带来新的技术。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

In recent years, there has been increasing effort to integrate modern artificial intelligence technologies into adaptive learning systems to enhance student learning. One key emerging area is in the use of models that can recognize student emotion in context, referred to as affective states. These models typically take the form of machine learning classifiers that recognize affect from the student's interaction with an online learning system. In this project, the investigators will develop adaptive learning systems that actively enlist the help of teachers to develop better student affect detection methods. In return, the system will support the work of teachers by providing them reports on the affective state of each student in real-time. The system will then learn to mimic teachers' choices of intervention methods for disengaged students in order to deliver interventions automatically. Overall, this project is anticipated to lead to i) better understanding of how to leverage and align to teachers' perspectives in detecting and responding to affect, and ii) enhanced intervention by both teachers and automated software that re-engages students and improves learning outcomes.This project will be organized into three phases. First, the investigators will employ active machine learning methods to ask teachers to observe specific students when they have a break in classroom activity; these methods can improve the quality of the affect detectors by providing data on the students whose affective states are most informative to improve the classifier, rather than the standard method of developing these detectors by observing students in round-robin fashion. Second, the investigators will incorporate richer data types (specifically, self-reported confidence ratings of affect labels) into the detectors to improve their quality. These self-reported confidence ratings reflect how uncertain humans are about specific affect judgements, which will be compared to the uncertainty of classifiers, to possibly reveal insights into student affect, such as what the properties are of situations where affect is ambiguous. Third, the investigators will use crowdsourcing to solicit ideas from teachers as to when specific affect interventions will be appropriate for specific students, and will develop automated intervention methods using reinforcement learning. These automated intervention methods are highly scalable since they can enable the system to take the actions the teacher would take to intervene to support different students experiencing negative affect at the same time. This intervention system will be tested in real classrooms as students learn within ASSISTments, a free web-based learning platform used by over 60,000 students a year. If successful, this project will lead to new scientific discoveries on the dynamics of affect and new technology for scalable student affect detection and intervention.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.