EAGER: Nonintrusive Engagement and Posture Detection in Virtual Classroom Environments

由于其便利性和节省时间的潜力，虚拟课程和其他在线活动在大流行后的世界仍然很受欢迎。不幸的是，虚拟出席的一个主要缺点是难以评估参与者的参与度，因此难以定制内容交付和解决特定的参与者问题。这个急切的项目的目的是探索非侵入性成像和基于音频的监测是否可以可靠地测量与敬业度评估相关的物理和生理参数。这样的评估可以为每个参与者私下进行，并可以形成对参与者和教师/领导者的谨慎反馈的基础。这涉及到与各种测量的准确在线分析相关的挑战，例如面部表情、眼睛凝视、姿势、身体运动和生理属性(例如心率、呼吸频率)。进一步的挑战包括解决个体差异和固有的“噪音”(即，与在线参与无关的身体和生理测量的变化)。因此，该项目的关键智力价值在于探索深度学习和基于逻辑推理的方法的新组合，以估计和组合(融合)这些措施，以确定如何准确和一致地使用远程监控来评估参与度并帮助改善虚拟学习环境。该项目的关键更广泛的影响在于其潜在的潜力，使在线学习环境更丰富、更高效，甚至可能超过非虚拟环境中人类直接感知的限制，同时保留虚拟参与的固有优势。如果成功，探索的方法有可能改变虚拟学习/会议环境中教师-学生/领导者-参与者的互动动态。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual classes and other online engagements continue to remain popular in the post-pandemic world because of their convenience and time-saving potential. Unfortunately, a major drawback of virtual attendance is the difficulty in assessing participant engagement, and hence, difficulty in tailoring content delivery and addressing specific participant issues. The purpose of this EAGER project is to explore whether nonintrusive imaging and audio-based monitoring can reliably gauge physical and physiological parameters relevant to engagement assessment. Such an assessment can be done privately for each participant and can form the basis for discreet feedback, both to the participant and the teacher/leader. This involves challenges associated with the accurate, online analysis of various measures, such as facial expressions, eye gaze, posture, body movements, and physiological attributes (e.g., heart rate, breathing rate). Further challenges include addressing individual variabilities and inherent “noise” (i.e., changes in physical and physiological measures unrelated to the online participation). Thus, the key intellectual merit of the project is in exploring novel combinations of deep learning and logic reasoning-based methods for estimating and combining (fusing) these measures to establish how accurately and consistently remote monitoring can be used to assess engagement and help improve virtual learning environments.The key broader impact of the project lies in its potential to make online learning environments richer and more productive, perhaps even exceeding the limits of direct human perception in nonvirtual settings, while preserving the inherent advantages of virtual participation. If successful, the explored methods have the potential to transform the dynamics of teacher-student/leader-participant interactions in virtual learning/meeting environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.