Detecting student engagement and designing pedagogical intervention in online learning

在线学习环境旨在提供有效的指导，但很少考虑到在有效学习中发挥重要作用的激励因素，如参与。标准的方法，如自我报告和观察清单评估，通过问卷调查，学习者的注意力，分心，兴奋，或无聊的学习活动的报告水平。然而，这些方法需要学习者和观察者付出相当多的时间和精力，并且还可能存在有效性问题，因为结果取决于学习者报告他们的情绪的意愿和他们对情绪的看法的准确性。虽然这些方法在教室环境中广泛使用，但在在线学习环境中并不有效。 本研究旨在设计一个能自动侦测学习者参与度的系统，并提供教学上的介入。认知科学和心理学概念以及计算机视觉和人工智能技术将用于评估学习者解释面部表情，手势和姿势的参与程度，并根据这些信息为在线学习者设计教学干预。具体而言，我们的三个目标如下： 设计一个新的学习者参与度检测系统，该系统考虑参与度水平、强度和报告频率、真实情绪以及与任务绩效的相关性 开发一个情感教学代理居住在学习环境中，可以适当地响应在线学习者的参与情感状态。 在现实生活中的在线学习环境中集成和评估参与检测和教学代理的性能。 研究表明，参与是可塑性的，适当的教学干预，学习设计和反馈可以提高学习者的参与。这项研究将采用创新方法评估这种参与，并有可能改变目前使用的标准方法。这项研究的高回报将在许多在线学习者和教师中发现，他们将从自动参与检测和个性化教学支持中受益。这些将产生显著的学习收益，提高生产力，并提高在线课程的保留率。 该项目将进一步扩展到使用多模式方法的学生参与检测和教学干预设计。来自面部表情、手势和姿势的信息将结合到学生的在线活动和课程论坛帖子中。我们还将研究增强现实和虚拟现实技术，以改善教学代理。我们希望本研究最终能有助于实现我们的长期目标，即为在线学习者开发一个有效的参与检测和教学干预设计框架。

Online learning environments aim to deliver efficacious instructions, but rarely take into consideration motivational factors like engagement that plays an important role in effective learning. Standard methods such as self-reporting and observational checklists assess, through questionnaires, learners' reported levels of attention, distraction, excitement, or boredom in their learning activities. However, these methods require considerable time and effort from both learners and observers and may also present issues with validity since the results depend on learners willingness to report their emotions and the accuracy of their perceptions about their emotions. While widely used in class-room settings, these methods are not effective in online learning settings. The purpose of this research is to design a system for automatic detection of learners' engagement and provide pedagogical intervention during online learning. Cognitive science and psychology concepts, as well as computer vision and artificial intelligence techniques will be used to assess learners' level of engagement interpreting facial expressions, gesture, and posture, and design pedagogical intervention based on that information for the online learners. Specifically, our three objectives are as follows: Design a new system for learner's engagement detection, one that considers engagement levels, intensity, and frequency of reports, authentic emotions, and correlations with task performance Develop an affective pedagogical agent to inhabit in the learning environment that can respond appropriately to affective states of engagement of the online learners. Integrate and evaluate the performance of engagement detection and pedagogical agent in a real-life online learning setting. Research shows that engagement is malleable and proper pedagogical interventions, learning designs, and feedback can enhance learner engagement. This research will take an innovative approach to assessments of such engagement, with the potential to shift the standard approaches currently in use. The high reward of this research will be found in the many stakeholdersa broad range of online learners and instructorswho will benefit from automatic engagement detection and personalized pedagogical support. These will generate significant learning gains, increased productivity, and higher retention rates in online courses. This project will be further extended for the students' engagement detection and pedagogical intervention design using multi-modal approaches. Information from facial expression, gesture, and posture will be combined to students' online activities and the course forum posts. We will also investigate augmented reality and virtual reality techniques to improve the pedagogical agents. We expect that this research will eventually contribute to achieving our long-term goal of developing an effective framework of engagement detection and pedagogical intervention design for the online learners.