INT: Collaborative Research: Detecting, Predicting and Remediating Student Affect and Grit Using Computer Vision

网络学习和未来学习技术计划为支持设想学习技术的未来并推进我们对人们如何在技术丰富的环境中学习的了解的努力提供资金。 整合（INT）项目完善和研究新兴的学习技术类型，这些技术已经在人们如何使用这些技术学习的严格研究背景下经历了几年的迭代完善; INT项目有助于我们理解原型工具如何推广到更大类别的学习技术。这个INT项目整合了两个发展良好的NSF赞助项目的先前工作，（i）先进的计算机视觉和（ii）智能辅导系统中的影响检测。 后者的工作，特别是开发工具，以检测学生的情绪（兴趣，困惑，挫折和无聊），并表明，当一个计算机导师回应消极的学生影响，学习成绩提高。目前的项目将把这一关注点扩展到情感之外，试图检测持久性、自我效能感和被称为“毅力”的特质。“该项目将衡量这些结构对学生学习的影响，并探讨是否可以改善砂砾特质（持续主动性和兴趣的持续倾向），以及它是否以及如何取决于最近经历的其他情绪。使这项研究成为广泛情感意识教学类型的技术创新是Smartutors，这是一种使用先进的计算机视觉技术来查看学生的凝视，手势，头部和面部的工具，以增加自动检测其影响的“带宽”。该研究团队汇集了计算机视觉、自适应教育技术和计算机科学、数学教育、学习同伴、元认知、情感、自我效能和动机等领域领先的跨学科研究人员。九个实验将提供有价值的数据来扩展和验证现有的坚毅和情感模型。特别是，该团队将收集关于毅力的细粒度数据，实时评估导师干预的影响，从而为毅力理论做出贡献。学生行为的可视化数据将与基于10，000多名先前学生行为的学生行为日志数据的高级分析相结合（例如，提示请求、主题掌握），以及时提供个性化指导和导师响应。这将使研究人员能够衡量干预措施对学生表现和态度的影响，并揭示坚毅水平与情绪的关系，以及情绪和坚毅对学生整体主动性的影响。通过识别对个体差异敏感的干预措施，本研究将完善动机和情感理论，并将揭示如何应对学生毅力和情感的原则，特别是当注意力和毅力开始减弱时。为了确保课堂上的成功，PI将与1，600名学生一起评估Smartutors，并通过与年龄较大的学生在更困难的数学领域进行测试来探索其可转移性。

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Integration (INT) projects refine and study emerging genres of learning technologies that have already undergone several years of iterative refinement in the context of rigorous research on how people learn with such technologies; INT projects contribute to our understanding of how the prototype tools might generalize to a larger category of learning technologies. This INT project integrates prior work from two well-developed NSF-sponsored projects on (i) advanced computer vision and (ii) affect detection in intelligent tutoring systems. The latter work in particular developed instruments to detect student emotion (interest, confusion, frustration and boredom) and showed that when a computer tutor responded to negative student affect, learning performance improved. The current project will expand this focus beyond emotion to attempt to also detect persistence, self-efficacy, and the trait called 'grit.' The project will measure the impact of these constructs on student learning and explore whether the grit trait (a persistent tendency towards sustained initiative and interest) can be improved and whether and how it depends on other recently experienced emotions. The technological innovation enabling this research into the genre of broadly affectively aware instruction is Smartutors, a tool that uses advanced computer vision techniques to view a student's gaze, hand gestures, head, and face to increase the "bandwidth" for automatically detecting their affect. One goal is to reorient students to more productive attitudes once waning attention is recognized.This research team brings together a unique blend of leading interdisciplinary researchers in computer vision; adaptive education technology and computer science; mathematics education; learning companions; and meta-cognition, emotion, self-efficacy and motivation. Nine experiments will provide valuable data to extend and validate existing models of grit and emotion. In particular, the team will gather fine-grained data on grit, assess the impact of tutor interventions in real-time, and contribute thereby to a theory of grit. Visual data of student behavior will be integrated with advanced analytics of log data of students' actions based on the behavior of over 10,000 prior students (e.g., hint requests, topic mastery) to provide individualized guidance and tutor responses in a timely fashion. This will allow the researchers to measure the impact of interventions on student performance and attitude, and it will uncover how grit levels relate to emotion and what impact emotions and grit combined have on overall student initiative. By identifying interventions that are sensitive to individual differences, this research will refine theories of motivation and emotion and will reveal principles about how to respond to student grit and affect, especially when attention and persistence begin to wane. To ensure classroom success, the PIs will evaluate Smartutors with 1,600 students and explore its transferability by testing it in a more difficult mathematics domain with older students.