INT: Collaborative Research: Detecting, Predicting and Remediating Student Affect and Grit Using Computer Vision

点击翻译按钮获取中文摘要

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Integration (INT) projects refine and study emerging genres of learning technologies that have already undergone several years of iterative refinement in the context of rigorous research on how people learn with such technologies; INT projects contribute to our understanding of how the prototype tools might generalize to a larger category of learning technologies. This INT project integrates prior work from two well-developed NSF-sponsored projects on (i) advanced computer vision and (ii) affect detection in intelligent tutoring systems. The latter work in particular developed instruments to detect student emotion (interest, confusion, frustration and boredom) and showed that when a computer tutor responded to negative student affect, learning performance improved. The current project will expand this focus beyond emotion to attempt to also detect persistence, self-efficacy, and the trait called 'grit.' The project will measure the impact of these constructs on student learning and explore whether the grit trait (a persistent tendency towards sustained initiative and interest) can be improved and whether and how it depends on other recently experienced emotions. The technological innovation enabling this research into the genre of broadly affectively aware instruction is Smartutors, a tool that uses advanced computer vision techniques to view a student's gaze, hand gestures, head, and face to increase the "bandwidth" for automatically detecting their affect. One goal is to reorient students to more productive attitudes once waning attention is recognized.This research team brings together a unique blend of leading interdisciplinary researchers in computer vision; adaptive education technology and computer science; mathematics education; learning companions; and meta-cognition, emotion, self-efficacy and motivation. Nine experiments will provide valuable data to extend and validate existing models of grit and emotion. In particular, the team will gather fine-grained data on grit, assess the impact of tutor interventions in real-time, and contribute thereby to a theory of grit. Visual data of student behavior will be integrated with advanced analytics of log data of students' actions based on the behavior of over 10,000 prior students (e.g., hint requests, topic mastery) to provide individualized guidance and tutor responses in a timely fashion. This will allow the researchers to measure the impact of interventions on student performance and attitude, and it will uncover how grit levels relate to emotion and what impact emotions and grit combined have on overall student initiative. By identifying interventions that are sensitive to individual differences, this research will refine theories of motivation and emotion and will reveal principles about how to respond to student grit and affect, especially when attention and persistence begin to wane. To ensure classroom success, the PIs will evaluate Smartutors with 1,600 students and explore its transferability by testing it in a more difficult mathematics domain with older students.