Investigating the Role of Interest in Middle Grade Science with a Multimodal Affect-Sensitive Learning Environment

学生的兴趣和影响在塑造个人学习方式方面发挥着关键作用。兴趣影响学生参与科学思想和实践，学生的学习成果，以及设定目标和自我调节的能力。学生的兴趣也是STEM职业发展的一个众所周知的先驱。在过去的几年里，对情感敏感的学习环境的研究已经使数据丰富的调查到学生学习的情感动力。然而，设计适应性学习环境，有效地响应学生的影响是研究中的一个关键差距。多模态影响识别和自适应学习技术的最新发展为创建情感响应干预措施奠定了基础，以支持学生的学习，参与和批判性地发展科学兴趣。本研究旨在设计、开发和研究一个多模式的情感敏感学习环境，以提高中学生的科学学习、参与和对科学的兴趣。该项目将调查学生对科学的影响和兴趣之间的关系，使方法的发展，以支持学习和兴趣的发展，通过多模式，影响敏感的干预措施，在一个以探究为基础的科学学习环境。预计该项目将推进为所有学生提供有效的、引人入胜的科学学习体验的国家目标。该项目的总体目标是开发能够改善STEM教育的方法和适应性学习技术，该项目有两个主要目标：第一个目标是设计，开发，并完善基于多模态神经架构的情感敏感学习环境，以产生和维持学生对基于探究的科学学习的兴趣。一套物理硬件传感器，用于捕获丰富的多通道数据（面部表情、眼睛注视、姿势、手势、交互跟踪日志），并结合对学生影响和行为参与的定量观察（即，已建立的观察协议）将被用来训练基于多模态递归神经网络的学生情感识别模型。这些模型将推动适应性干预，引导学生通过触发和保持学生对科学探究的兴趣来解决问题。对影响敏感的干预措施将与水晶岛相结合，水晶岛是一个以探究为基础的中学科学教育学习环境。第二个目标是调查的影响，多模态的情感敏感的学习环境对学生的学习，参与，和科学的兴趣。一个最终的研究与中学生将检查这些设计的影响，比较多模态影响敏感的学习环境的基线环境，没有影响敏感的干预措施。这种比较将测试学习环境在促进不同学习者增强学习和兴趣成果方面的有效性，检查知识，参与和兴趣的措施，包括对科学的兴趣和对STEM职业的兴趣。由此产生的研究结果将产生重大贡献的理论和实践在学生的兴趣发展和生产的有效性的多模态神经架构的建模和响应studentaffect.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Student interest and affect play critical roles in shaping how individuals learn. Interest impacts student engagement with scientific ideas and practices, student learning outcomes, and the ability to set goals and self-regulate. Student interest is also a well-known precursor to STEM career development. Over the past several years, research on affect-sensitive learning environments has enabled data-rich investigations into the affective dynamics of student learning. However, designing adaptive learning environments that respond effectively to student affect is a key gap in the research. Recent developments in multimodal affect recognition and adaptive learning technologies have set the stage for the creation of affect-responsive interventions to support student learning, engagement, and critically, the development of science interest. This project centers on the design, development, and investigation of a multimodal affect-sensitive learning environment to enhance middle school students’ science learning, engagement, and interest in science. The project will investigate the relationship between student affect and interest in science, enabling the development of methods to support learning and interest development through multimodal, affect-sensitive interventions within an inquiry-based science learning environment. It is anticipated that the project will advance the national goal of providing effective, engaging science learning experiences for all students.With the overarching goal of developing methods and adaptive learning technologies that enable improved STEM education, the project has two major objectives: The first objective is to design, develop, and refine an affect-sensitive learning environment based on multimodal neural architectures to generate and sustain student interest in inquiry-based science learning. A suite of physical hardware sensors to capture rich multi-channel data (facial expression, eye gaze, posture, gesture, interaction trace logs) combined with quantitative observations of student affect and behavioral engagement (i.e., an established protocol for observations) will be utilized to train multimodal recurrent neural network-based models of student affect recognition. These models will drive adaptive interventions to guide students toward engaged problem solving by triggering and maintaining student interest in science inquiry. The affect-sensitive interventions will be integrated with Crystal Island, an inquiry-based learning environment for middle school science education. The second objective is to investigate the impact of the multimodal affect-sensitive learning environment on student learning, engagement, and interest in science. A culminating study with middle school students will examine the impact of these designs, comparing the multimodal affect-sensitive learning environment to a baseline environment without affect-sensitive interventions. This comparison will test the effectiveness of the learning environment in fostering enhanced learning and interest outcomes across a diverse range of learners, examining measures of knowledge, engagement, and interest, including interest in science and interest in STEM careers. The resulting findings will yield significant contributions to both theory and practice in student interest development and produce an empirical account of the effectiveness of multimodal neural architectures for modeling and responding to student affect.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.