INT: Collaborative Research: Detecting, Predicting and Remediating Student Affect and Grit Using Computer Vision

“网络学习和未来学习技术计划”资助的工作旨在支持展望学习技术的未来，并推进我们对人们如何在技术丰富的环境中学习的了解。集成（INT）项目细化和研究新兴类型的学习技术，这些技术已经在严格研究人们如何使用这些技术进行学习的背景下经历了几年的迭代细化；INT项目有助于我们理解原型工具如何推广到更大的学习技术类别。这个INT项目整合了两个发达的nsf资助项目的先前工作，分别是(i)先进计算机视觉和（ii）智能辅导系统中的影响检测。后一项工作特别开发了检测学生情绪（兴趣、困惑、沮丧和无聊）的工具，并表明当计算机导师对学生的负面情绪做出反应时，学习成绩会有所提高。目前的项目将把这一重点扩展到情感之外，试图检测持久性、自我效能和“毅力”特质。该项目将测量这些构念对学生学习的影响，并探索毅力特质（一种持续的主动性和兴趣的持续倾向）是否可以改善，以及它是否以及如何依赖于其他最近经历的情绪。使这项研究进入广泛情感感知教学类型的技术创新是Smartutors，这是一种使用先进的计算机视觉技术来观察学生的目光、手势、头部和面部的工具，以增加自动检测他们的情感的“带宽”。其中一个目标是，一旦意识到注意力的减弱，让学生重新定位到更有成效的态度。该研究团队汇集了计算机视觉领域领先的跨学科研究人员的独特融合；适应性教育技术与计算机科学；数学教育;最好的学习伙伴;元认知，情感，自我效能和动机。九个实验将提供有价值的数据来扩展和验证现有的勇气和情感模型。特别是，该团队将收集关于砂砾的细粒度数据，实时评估导师干预的影响，从而为砂砾理论做出贡献。学生行为的可视化数据将与基于10000多名在先学生行为（如提示请求、主题掌握）的学生行为日志数据的高级分析相结合，及时提供个性化指导和导师回应。这将使研究人员能够衡量干预措施对学生表现和态度的影响，并将揭示毅力水平与情绪的关系，以及情绪和毅力结合起来对学生整体主动性的影响。通过确定对个体差异敏感的干预措施，这项研究将完善动机和情感的理论，并揭示如何应对学生的勇气和影响的原则，特别是当注意力和毅力开始减弱时。为了确保课堂上的成功，pi将对1600名学生进行评估，并通过对年龄较大的学生在更难的数学领域进行测试来探索其可移植性。

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Integration (INT) projects refine and study emerging genres of learning technologies that have already undergone several years of iterative refinement in the context of rigorous research on how people learn with such technologies; INT projects contribute to our understanding of how the prototype tools might generalize to a larger category of learning technologies. This INT project integrates prior work from two well-developed NSF-sponsored projects on (i) advanced computer vision and (ii) affect detection in intelligent tutoring systems. The latter work in particular developed instruments to detect student emotion (interest, confusion, frustration and boredom) and showed that when a computer tutor responded to negative student affect, learning performance improved. The current project will expand this focus beyond emotion to attempt to also detect persistence, self-efficacy, and the trait called 'grit.' The project will measure the impact of these constructs on student learning and explore whether the grit trait (a persistent tendency towards sustained initiative and interest) can be improved and whether and how it depends on other recently experienced emotions. The technological innovation enabling this research into the genre of broadly affectively aware instruction is Smartutors, a tool that uses advanced computer vision techniques to view a student's gaze, hand gestures, head, and face to increase the "bandwidth" for automatically detecting their affect. One goal is to reorient students to more productive attitudes once waning attention is recognized.This research team brings together a unique blend of leading interdisciplinary researchers in computer vision; adaptive education technology and computer science; mathematics education; learning companions; and meta-cognition, emotion, self-efficacy and motivation. Nine experiments will provide valuable data to extend and validate existing models of grit and emotion. In particular, the team will gather fine-grained data on grit, assess the impact of tutor interventions in real-time, and contribute thereby to a theory of grit. Visual data of student behavior will be integrated with advanced analytics of log data of students' actions based on the behavior of over 10,000 prior students (e.g., hint requests, topic mastery) to provide individualized guidance and tutor responses in a timely fashion. This will allow the researchers to measure the impact of interventions on student performance and attitude, and it will uncover how grit levels relate to emotion and what impact emotions and grit combined have on overall student initiative. By identifying interventions that are sensitive to individual differences, this research will refine theories of motivation and emotion and will reveal principles about how to respond to student grit and affect, especially when attention and persistence begin to wane. To ensure classroom success, the PIs will evaluate Smartutors with 1,600 students and explore its transferability by testing it in a more difficult mathematics domain with older students.