EAGER: Nonintrusive Engagement and Posture Detection in Virtual Classroom Environments

虚拟课程和其他在线活动在大流行后的世界继续流行，因为它们方便和节省时间。不幸的是，虚拟出席的一个主要缺点是难以评估参与者的参与度，因此，难以定制内容交付和解决特定的参与者问题。这个EAGER项目的目的是探索非侵入式成像和基于音频的监测是否可以可靠地测量与交战评估相关的物理和生理参数。这样的评估可以私下对每个参与者进行，并且可以形成对参与者和老师/领导的谨慎反馈的基础。这涉及到与各种测量的准确在线分析相关的挑战，例如面部表情、目光、姿势、身体运动和生理属性（例如，心率、呼吸频率）。进一步的挑战包括解决个体差异和固有的“噪音”（即与在线参与无关的生理和生理指标的变化）。因此，该项目的关键智力价值在于探索深度学习和基于逻辑推理的方法的新组合，以评估和组合（融合）这些措施，以确定如何准确和一致地使用远程监控来评估参与度并帮助改善虚拟学习环境。该项目更广泛的关键影响在于，它有可能使在线学习环境更丰富、更富有成效，甚至可能超越人类在非虚拟环境中直接感知的极限，同时保留虚拟参与的固有优势。如果成功，所探索的方法有可能改变虚拟学习/会议环境中教师-学生/领导者-参与者互动的动态。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Virtual classes and other online engagements continue to remain popular in the post-pandemic world because of their convenience and time-saving potential. Unfortunately, a major drawback of virtual attendance is the difficulty in assessing participant engagement, and hence, difficulty in tailoring content delivery and addressing specific participant issues. The purpose of this EAGER project is to explore whether nonintrusive imaging and audio-based monitoring can reliably gauge physical and physiological parameters relevant to engagement assessment. Such an assessment can be done privately for each participant and can form the basis for discreet feedback, both to the participant and the teacher/leader. This involves challenges associated with the accurate, online analysis of various measures, such as facial expressions, eye gaze, posture, body movements, and physiological attributes (e.g., heart rate, breathing rate). Further challenges include addressing individual variabilities and inherent “noise” (i.e., changes in physical and physiological measures unrelated to the online participation). Thus, the key intellectual merit of the project is in exploring novel combinations of deep learning and logic reasoning-based methods for estimating and combining (fusing) these measures to establish how accurately and consistently remote monitoring can be used to assess engagement and help improve virtual learning environments.The key broader impact of the project lies in its potential to make online learning environments richer and more productive, perhaps even exceeding the limits of direct human perception in nonvirtual settings, while preserving the inherent advantages of virtual participation. If successful, the explored methods have the potential to transform the dynamics of teacher-student/leader-participant interactions in virtual learning/meeting environments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.