INT: Collaborative Research: Detecting, Predicting and Remediating Student Affect and Grit Using Computer Vision

INT：协作研究：使用计算机视觉检测、预测和纠正学生的情绪和毅力

• 批准号: 2104984
• 负责人: Ivon Arroyo
• 金额: $ 8.03万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2104984&HistoricalAwards=false
• 关键词: INT; Collaborative; Research; Detecting; Predicting

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Integration (INT) projects refine and study emerging genres of learning technologies that have already undergone several years of iterative refinement in the context of rigorous research on how people learn with such technologies; INT projects contribute to our understanding of how the prototype tools might generalize to a larger category of learning technologies. This INT project integrates prior work from two well-developed NSF-sponsored projects on (i) advanced computer vision and (ii) affect detection in intelligent tutoring systems. The latter work in particular developed instruments to detect student emotion (interest, confusion, frustration and boredom) and showed that when a computer tutor responded to negative student affect, learning performance improved. The current project will expand this focus beyond emotion to attempt to also detect persistence, self-efficacy, and the trait called 'grit.' The project will measure the impact of these constructs on student learning and explore whether the grit trait (a persistent tendency towards sustained initiative and interest) can be improved and whether and how it depends on other recently experienced emotions. The technological innovation enabling this research into the genre of broadly affectively aware instruction is Smartutors, a tool that uses advanced computer vision techniques to view a student's gaze, hand gestures, head, and face to increase the "bandwidth" for automatically detecting their affect. One goal is to reorient students to more productive attitudes once waning attention is recognized.This research team brings together a unique blend of leading interdisciplinary researchers in computer vision; adaptive education technology and computer science; mathematics education; learning companions; and meta-cognition, emotion, self-efficacy and motivation. Nine experiments will provide valuable data to extend and validate existing models of grit and emotion. In particular, the team will gather fine-grained data on grit, assess the impact of tutor interventions in real-time, and contribute thereby to a theory of grit. Visual data of student behavior will be integrated with advanced analytics of log data of students' actions based on the behavior of over 10,000 prior students (e.g., hint requests, topic mastery) to provide individualized guidance and tutor responses in a timely fashion. This will allow the researchers to measure the impact of interventions on student performance and attitude, and it will uncover how grit levels relate to emotion and what impact emotions and grit combined have on overall student initiative. By identifying interventions that are sensitive to individual differences, this research will refine theories of motivation and emotion and will reveal principles about how to respond to student grit and affect, especially when attention and persistence begin to wane. To ensure classroom success, the PIs will evaluate Smartutors with 1,600 students and explore its transferability by testing it in a more difficult mathematics domain with older students.

网络学习和未来学习技术计划为支持展望学习技术的未来并推动我们了解人们如何在技术丰富的环境中学习的努力提供资金。整合(INT)项目在严格研究人们如何使用这些技术学习的背景下，精炼和研究新兴的学习技术流派，这些技术已经经历了几年的迭代完善；INT项目有助于我们理解原型工具如何推广到更大类别的学习技术。这个INT项目整合了NSF赞助的两个项目的先前工作，这些项目是关于(I)先进的计算机视觉和(Ii)智能教学系统中的情感检测。后者的工作特别是开发了检测学生情绪(兴趣、困惑、沮丧和无聊)的工具，并表明当电脑导师对学生的负面情绪做出反应时，学习成绩会有所改善。目前的项目将把这种关注扩展到情感之外，试图同时检测坚持性、自我效能感和一种称为“坚韧不拔”的特质。该项目将衡量这些结构对学生学习的影响，并探索坚韧特质(对持续的主动性和兴趣的持续倾向)是否可以改善，以及它是否以及如何依赖于最近经历的其他情绪。这项研究的技术创新是Smartutors，这是一种使用先进的计算机视觉技术来观察学生的凝视、手势、头部和面部的工具，以增加自动检测他们的情感的“带宽”。一个目标是一旦意识到注意力减弱，就将学生重新引导到更有成效的态度。这个研究团队汇集了计算机视觉、适应性教育技术和计算机科学、数学教育、学习伙伴以及元认知、情感、自我效能和动机等领域的领先跨学科研究人员的独特组合。九个实验将提供有价值的数据，以扩展和验证现有的砂砾和情感模型。特别是，该团队将收集关于GRIT的细粒度数据，实时评估导师干预的影响，从而为GRIT理论做出贡献。学生行为的可视数据将与基于10,000多名先前学生的行为(例如，提示请求、主题掌握)的学生行为日志数据的高级分析相结合，以及时提供个性化指导和导师响应。这将使研究人员能够衡量干预措施对学生表现和态度的影响，并揭示坚韧程度如何与情感相关，以及情感和坚韧结合在一起对学生的整体主动性产生了什么影响。通过确定对个体差异敏感的干预措施，这项研究将完善动机和情绪理论，并揭示如何应对学生的毅力和情感的原则，特别是在注意力和毅力开始减弱的情况下。为了确保课堂成功，PI将评估1,600名学生的Smartutors，并通过在更困难的数学领域与年龄较大的学生一起测试来探索其可转移性。