Investigating the Role of Interest in Middle Grade Science with a Multimodal Affect-Sensitive Learning Environment

学生的兴趣和影响在塑造个人的学习方式中起着至关重要的作用。兴趣影响学生对科学思想和实践的参与，学生的学习成果，以及设定目标和自我调节的能力。众所周知，学生的兴趣也是STEM职业发展的前兆。在过去的几年里，对情感敏感学习环境的研究使得对学生学习的情感动态进行了数据丰富的调查。然而，如何设计能有效响应学生情感的适应性学习环境是研究的一个关键空白。多模态情感识别和适应性学习技术的最新发展为情感响应干预的创建奠定了基础，以支持学生的学习、参与，以及关键的科学兴趣的发展。本项目旨在设计、开发和调查一个多模态情感敏感学习环境，以提高中学生的科学学习、参与和兴趣。该项目将调查学生情感与科学兴趣之间的关系，在基于探究的科学学习环境中，通过多模式、情感敏感的干预措施，开发支持学习和兴趣发展的方法。预计该项目将推进为所有学生提供有效、引人入胜的科学学习体验的国家目标。该项目的总体目标是开发方法和自适应学习技术，以改善STEM教育，该项目有两个主要目标：第一个目标是设计、开发和完善基于多模态神经架构的影响敏感学习环境，以产生和维持学生对基于探究的科学学习的兴趣。一套物理硬件传感器用于捕获丰富的多通道数据（面部表情、眼神、姿势、手势、交互跟踪日志），结合学生情感和行为参与的定量观察（即建立的观察协议），将用于训练基于多模态递归神经网络的学生情感识别模型。这些模型将推动适应性干预，通过激发和保持学生对科学探究的兴趣，引导学生参与解决问题。情感敏感干预将与水晶岛中学科学教育的探究式学习环境相结合。第二个目标是调查多模态情感敏感学习环境对学生学习、参与和科学兴趣的影响。最后一项针对中学生的研究将检验这些设计的影响，将多模态情感敏感学习环境与没有情感敏感干预的基线环境进行比较。这种比较将测试学习环境在促进不同学习者的学习和兴趣成果方面的有效性，检查知识、参与度和兴趣的衡量标准，包括对科学的兴趣和对STEM职业的兴趣。由此产生的研究结果将对学生兴趣发展的理论和实践做出重大贡献，并对多模态神经结构在建模和响应学生情感方面的有效性进行实证说明。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Student interest and affect play critical roles in shaping how individuals learn. Interest impacts student engagement with scientific ideas and practices, student learning outcomes, and the ability to set goals and self-regulate. Student interest is also a well-known precursor to STEM career development. Over the past several years, research on affect-sensitive learning environments has enabled data-rich investigations into the affective dynamics of student learning. However, designing adaptive learning environments that respond effectively to student affect is a key gap in the research. Recent developments in multimodal affect recognition and adaptive learning technologies have set the stage for the creation of affect-responsive interventions to support student learning, engagement, and critically, the development of science interest. This project centers on the design, development, and investigation of a multimodal affect-sensitive learning environment to enhance middle school students’ science learning, engagement, and interest in science. The project will investigate the relationship between student affect and interest in science, enabling the development of methods to support learning and interest development through multimodal, affect-sensitive interventions within an inquiry-based science learning environment. It is anticipated that the project will advance the national goal of providing effective, engaging science learning experiences for all students.With the overarching goal of developing methods and adaptive learning technologies that enable improved STEM education, the project has two major objectives: The first objective is to design, develop, and refine an affect-sensitive learning environment based on multimodal neural architectures to generate and sustain student interest in inquiry-based science learning. A suite of physical hardware sensors to capture rich multi-channel data (facial expression, eye gaze, posture, gesture, interaction trace logs) combined with quantitative observations of student affect and behavioral engagement (i.e., an established protocol for observations) will be utilized to train multimodal recurrent neural network-based models of student affect recognition. These models will drive adaptive interventions to guide students toward engaged problem solving by triggering and maintaining student interest in science inquiry. The affect-sensitive interventions will be integrated with Crystal Island, an inquiry-based learning environment for middle school science education. The second objective is to investigate the impact of the multimodal affect-sensitive learning environment on student learning, engagement, and interest in science. A culminating study with middle school students will examine the impact of these designs, comparing the multimodal affect-sensitive learning environment to a baseline environment without affect-sensitive interventions. This comparison will test the effectiveness of the learning environment in fostering enhanced learning and interest outcomes across a diverse range of learners, examining measures of knowledge, engagement, and interest, including interest in science and interest in STEM careers. The resulting findings will yield significant contributions to both theory and practice in student interest development and produce an empirical account of the effectiveness of multimodal neural architectures for modeling and responding to student affect.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.