Detecting student engagement and designing pedagogical intervention in online learning

在线学习环境旨在提供有效的指导，但很少考虑参与等激励因素，这些因素在有效学习中发挥着重要作用。标准方法，如自我报告和观察清单，通过问卷调查，评估学习者在学习活动中报告的注意力、分心、兴奋或无聊程度。然而，这些方法需要学习者和观察者花费大量的时间和精力，而且可能会带来有效性的问题，因为结果取决于学习者是否愿意报告他们的情绪以及他们对情绪的感知的准确性。虽然这些方法广泛应用于课堂环境中，但在在线学习环境中并不有效。 本研究的目的是设计一个自动检测学习者参与度的系统，并在在线学习过程中提供教学干预。将使用认知科学和心理学概念，以及计算机视觉和人工智能技术来评估学习者解读面部表情、手势和姿势的参与度，并根据这些信息为在线学习者设计教学干预措施。具体来说，我们的三个目标如下： 设计一个新的学习者敬业度检测系统，该系统考虑到学习者的敬业度、报告的强度和频率、真实的情绪以及与任务表现的相关性 开发一种情感教学代理，以居住在能够对在线学习者的情感投入状态做出适当反应的学习环境中。 在真实的在线学习环境中集成和评估参与度检测和教学代理的性能。 研究表明，参与度是可塑性的，适当的教学干预、学习设计和反馈可以提高学习者的参与度。这项研究将采用一种创新的方法来评估这种参与，并有可能改变目前使用的标准方法。这项研究的高额回报将在广泛的在线学习者和教师中发现，他们将受益于自动参与检测和个性化教学支持。这将带来显著的学习收益、更高的工作效率和更高的在线课程留存率。 该项目将进一步扩展到学生的参与检测和使用多模式方法的教学干预设计。来自面部表情、手势和姿势的信息将被结合到学生的在线活动和课程论坛帖子中。我们还将研究增强现实和虚拟现实技术，以改进教学代理。我们期望这项研究最终将有助于实现我们的长期目标，即为在线学习者开发一个有效的参与度检测和教学干预设计框架。

Online learning environments aim to deliver efficacious instructions, but rarely take into consideration motivational factors like engagement that plays an important role in effective learning. Standard methods such as self-reporting and observational checklists assess, through questionnaires, learners' reported levels of attention, distraction, excitement, or boredom in their learning activities. However, these methods require considerable time and effort from both learners and observers and may also present issues with validity since the results depend on learners willingness to report their emotions and the accuracy of their perceptions about their emotions. While widely used in class-room settings, these methods are not effective in online learning settings. The purpose of this research is to design a system for automatic detection of learners' engagement and provide pedagogical intervention during online learning. Cognitive science and psychology concepts, as well as computer vision and artificial intelligence techniques will be used to assess learners' level of engagement interpreting facial expressions, gesture, and posture, and design pedagogical intervention based on that information for the online learners. Specifically, our three objectives are as follows: Design a new system for learner's engagement detection, one that considers engagement levels, intensity, and frequency of reports, authentic emotions, and correlations with task performance Develop an affective pedagogical agent to inhabit in the learning environment that can respond appropriately to affective states of engagement of the online learners. Integrate and evaluate the performance of engagement detection and pedagogical agent in a real-life online learning setting. Research shows that engagement is malleable and proper pedagogical interventions, learning designs, and feedback can enhance learner engagement. This research will take an innovative approach to assessments of such engagement, with the potential to shift the standard approaches currently in use. The high reward of this research will be found in the many stakeholdersa broad range of online learners and instructorswho will benefit from automatic engagement detection and personalized pedagogical support. These will generate significant learning gains, increased productivity, and higher retention rates in online courses. This project will be further extended for the students' engagement detection and pedagogical intervention design using multi-modal approaches. Information from facial expression, gesture, and posture will be combined to students' online activities and the course forum posts. We will also investigate augmented reality and virtual reality techniques to improve the pedagogical agents. We expect that this research will eventually contribute to achieving our long-term goal of developing an effective framework of engagement detection and pedagogical intervention design for the online learners.