Investigating the Role of Interest in Middle Grade Science with a Multimodal Affect-Sensitive Learning Environment

学生的兴趣和情感在塑造个人学习方式方面起着至关重要的作用。兴趣影响学生对科学思想和实践的参与度、学生的学习结果以及设定目标和自我调节的能力。学生的兴趣也是STEM职业发展的一个众所周知的先兆。在过去的几年里，对情感敏感学习环境的研究使得对学生学习的情感动态的数据丰富的调查成为可能。然而，设计对学生情感做出有效反应的适应性学习环境是这项研究中的一个关键空白。多模式情感识别和适应性学习技术的最新发展为创建情感反应干预措施奠定了基础，以支持学生的学习、参与，尤其是科学兴趣的发展。本项目的核心是设计、开发和调查一个多通道情感敏感的学习环境，以提高中学生的科学学习、参与和对科学的兴趣。该项目将调查学生情感和对科学的兴趣之间的关系，从而能够在以探究为基础的科学学习环境中，通过多模式、对情感敏感的干预措施，制定支持学习和兴趣发展的方法。预计该项目将推进为所有学生提供有效、参与的科学学习体验的国家目标。该项目的总体目标是开发方法和适应性学习技术，以改进STEM教育，该项目有两个主要目标：第一个目标是设计、开发和优化基于多模式神经体系结构的情感敏感型学习环境，以激发和保持学生对探究性科学学习的兴趣。将利用一套物理硬件传感器来捕获丰富的多通道数据(面部表情、眼睛凝视、姿势、手势、交互跟踪日志)，并结合对学生情感和行为参与的定量观察(即，已建立的观察协议)来训练基于多模式递归神经网络的学生情感识别模型。这些模式将推动适应性干预，通过激发和保持学生对科学探究的兴趣，引导学生参与解决问题。这些情感敏感的干预措施将与水晶岛整合，水晶岛是一个针对中学科学教育的探究式学习环境。第二个目标是调查多通道情感敏感型学习环境对学生学习、参与和对科学的兴趣的影响。一项针对中学生的最终研究将检验这些设计的影响，将多模式情感敏感学习环境与没有情感敏感干预的基线环境进行比较。这一比较将测试学习环境在促进各种不同学习者的学习和兴趣成果方面的有效性，检查知识、参与度和兴趣的衡量标准，包括对科学的兴趣和对STEM职业的兴趣。其结果将对学生兴趣发展的理论和实践做出重大贡献，并对多通道神经结构在建模和响应学生影响方面的有效性进行实证说明。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Student interest and affect play critical roles in shaping how individuals learn. Interest impacts student engagement with scientific ideas and practices, student learning outcomes, and the ability to set goals and self-regulate. Student interest is also a well-known precursor to STEM career development. Over the past several years, research on affect-sensitive learning environments has enabled data-rich investigations into the affective dynamics of student learning. However, designing adaptive learning environments that respond effectively to student affect is a key gap in the research. Recent developments in multimodal affect recognition and adaptive learning technologies have set the stage for the creation of affect-responsive interventions to support student learning, engagement, and critically, the development of science interest. This project centers on the design, development, and investigation of a multimodal affect-sensitive learning environment to enhance middle school students’ science learning, engagement, and interest in science. The project will investigate the relationship between student affect and interest in science, enabling the development of methods to support learning and interest development through multimodal, affect-sensitive interventions within an inquiry-based science learning environment. It is anticipated that the project will advance the national goal of providing effective, engaging science learning experiences for all students.With the overarching goal of developing methods and adaptive learning technologies that enable improved STEM education, the project has two major objectives: The first objective is to design, develop, and refine an affect-sensitive learning environment based on multimodal neural architectures to generate and sustain student interest in inquiry-based science learning. A suite of physical hardware sensors to capture rich multi-channel data (facial expression, eye gaze, posture, gesture, interaction trace logs) combined with quantitative observations of student affect and behavioral engagement (i.e., an established protocol for observations) will be utilized to train multimodal recurrent neural network-based models of student affect recognition. These models will drive adaptive interventions to guide students toward engaged problem solving by triggering and maintaining student interest in science inquiry. The affect-sensitive interventions will be integrated with Crystal Island, an inquiry-based learning environment for middle school science education. The second objective is to investigate the impact of the multimodal affect-sensitive learning environment on student learning, engagement, and interest in science. A culminating study with middle school students will examine the impact of these designs, comparing the multimodal affect-sensitive learning environment to a baseline environment without affect-sensitive interventions. This comparison will test the effectiveness of the learning environment in fostering enhanced learning and interest outcomes across a diverse range of learners, examining measures of knowledge, engagement, and interest, including interest in science and interest in STEM careers. The resulting findings will yield significant contributions to both theory and practice in student interest development and produce an empirical account of the effectiveness of multimodal neural architectures for modeling and responding to student affect.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.