EAGER: Nonintrusive Engagement and Posture Detection in Virtual Classroom Environments

虚拟课程及其他网上活动因其便利及节省时间的潜力，在疫情后的世界继续受欢迎。不幸的是，虚拟出席的一个主要缺点是难以评估参与者的参与度，因此难以定制内容交付和解决特定的参与者问题。EAGER项目的目的是探索非侵入性成像和基于音频的监测是否可以可靠地测量与参与评估相关的物理和生理参数。这样的评估可以为每个参与者私下进行，并可以形成对参与者和教师/领导者的谨慎反馈的基础。这涉及与各种测量的准确在线分析相关联的挑战，所述测量诸如面部表情、眼睛注视、姿势、身体运动和生理属性（例如，心率、呼吸率）。进一步的挑战包括解决个体可变性和固有的“噪声”（即，与在线参与无关的身体和生理测量的变化）。因此，该项目的主要智力价值在于探索深度学习和基于逻辑推理的方法的新组合，用于估计和组合（融合）这些措施，以确定如何准确和一致地使用远程监控来评估参与度并帮助改善虚拟学习环境。该项目的主要广泛影响在于其可能使在线学习环境更丰富，更富有成效，也许甚至超过了人类在非虚拟环境中直接感知的极限，同时保留了虚拟参与的固有优势。如果成功的话，所探索的方法有可能改变教师-学生/领导-参与者在虚拟学习/会议环境中的互动动态。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Virtual classes and other online engagements continue to remain popular in the post-pandemic world because of their convenience and time-saving potential. Unfortunately, a major drawback of virtual attendance is the difficulty in assessing participant engagement, and hence, difficulty in tailoring content delivery and addressing specific participant issues. The purpose of this EAGER project is to explore whether nonintrusive imaging and audio-based monitoring can reliably gauge physical and physiological parameters relevant to engagement assessment. Such an assessment can be done privately for each participant and can form the basis for discreet feedback, both to the participant and the teacher/leader. This involves challenges associated with the accurate, online analysis of various measures, such as facial expressions, eye gaze, posture, body movements, and physiological attributes (e.g., heart rate, breathing rate). Further challenges include addressing individual variabilities and inherent “noise” (i.e., changes in physical and physiological measures unrelated to the online participation). Thus, the key intellectual merit of the project is in exploring novel combinations of deep learning and logic reasoning-based methods for estimating and combining (fusing) these measures to establish how accurately and consistently remote monitoring can be used to assess engagement and help improve virtual learning environments.The key broader impact of the project lies in its potential to make online learning environments richer and more productive, perhaps even exceeding the limits of direct human perception in nonvirtual settings, while preserving the inherent advantages of virtual participation. If successful, the explored methods have the potential to transform the dynamics of teacher-student/leader-participant interactions in virtual learning/meeting environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.