Trans-modal Analysis: A Mathematical and Computational Framework for Equity Assessment of Multi-modal STEM Learning Processes

点击翻译按钮获取中文摘要

In modern classrooms, students learn science, technology, engineering, and mathematics (STEM) through interactions not only with teachers and textbooks but also with computer games and simulations, automated tutors, and online resources. Education researchers thus have access to large amounts of data about students’ STEM learning processes, from classroom or online conversations to detailed records of student activity in educational apps. Despite the potential of such rich data for curriculum development and personalized assessment, there are significant technical and conceptual challenges to analyzing data that come from different sources or modalities. To address these challenges, this project will develop and test trans-modal analysis (TMA). TMA is a statistical technique and software package that will help researchers better and more easily integrate multiple types of data into analyses of STEM learning. This will enable more accurate understanding of students’ STEM learning processes, and in turn help identify potential inequities in assessment of student learning, informing education policy and practice for diverse learners. This project will include post-doctoral scholars, graduate student researchers, and undergraduate research interns, who will develop skills and experience in data science, learning analytics, software development, and scientific communication, providing training and mentoring for the next generation of education researchers. Although most analyses of learning processes are based on a single type or modality of data, STEM learning typically takes place in a multimodal setting. Models of STEM learning processes thus need to account for multiple sources and types of data to account for complex interactions between learners and the setting(s) in which they learn. For example, there are different types of events (questions from a teacher, chats with a peer, views of a resource) and different properties of events (gender of a person gesturing, linguistic fluency of a speaker, reading level of a person reading a document) that may influence future events with more or less impact over time. In addition, the structure of a learning environment creates a horizon of observation for each student, making some events (e.g., a conversation in another group of students) more or less visible. Finally, different characteristics of students (age, cultural or ethnic background, gender identification, whether instruction is in their native language or a non-native language) may lead them to respond to events in different ways. Extant learning analytic techniques account for the influence of prior events by lagging: for example, using some fixed number of prior events to predict future events. TMA will enable those same techniques to operate not on properties of the events themselves but on underlying functions that represent claims or hypotheses about the interaction between different learning modalities, the structure of the learning environment, and the ways in which students might systematically differ as STEM learners. The project team hypothesizes that TMA models will provide a more nuanced, more accurate, and more equitable view of STEM learning processes for diverse learners. This approach will expand the understanding of effective multi-modal STEM learning processes and allow researchers to account for diversity and address questions of equity in multi-modal STEM learning. TMA will be developed and tested first as a set of algorithms for conducting trans-modal analyses with three widely used learning analytic tools: process mining, epistemic network analysis, and dynamic Bayesian networks. The investigators aim to use simulation studies and the analysis of actual STEM learning datasets to address two fundamental research questions regarding the science of learning: (1) Under what conditions (if any) are trans-modal models of STEM learning processes more informative than uni-modal models? And (2) Can TMA model meaningful differences in trans-modal learning processes for minoritized groups of STEM learners?This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.