Trans-modal Analysis: A Mathematical and Computational Framework for Equity Assessment of Multi-modal STEM Learning Processes

在现代课堂上，学生学习科学，技术，工程和数学（STEM），不仅通过与教师和教科书的互动，还通过计算机游戏和模拟，自动导师和在线资源。因此，教育研究人员可以访问有关学生STEM学习过程的大量数据，从课堂或在线对话到教育应用程序中学生活动的详细记录。尽管如此丰富的数据对于课程开发和个性化评估具有潜力，但分析来自不同来源或模式的数据存在重大的技术和概念挑战。为了应对这些挑战，该项目将开发和测试跨模态分析（TMA）。TMA是一种统计技术和软件包，将帮助研究人员更好，更容易地将多种类型的数据整合到STEM学习分析中。这将有助于更准确地了解学生的STEM学习过程，进而有助于识别学生学习评估中的潜在不平等，为不同学习者的教育政策和实践提供信息。该项目将包括博士后学者，研究生研究人员和本科研究实习生，他们将发展数据科学，学习分析，软件开发和科学传播方面的技能和经验，为下一代教育研究人员提供培训和指导。虽然大多数学习过程的分析都是基于单一类型或模式的数据，但STEM学习通常发生在多模式环境中。因此，STEM学习过程的模型需要考虑多种来源和类型的数据，以考虑学习者与他们学习的环境之间的复杂互动。例如，存在不同类型的事件（来自老师的问题、与同伴的聊天、对资源的看法）和不同的事件属性（做手势的人的性别、说话者的语言流利度、阅读文档的人的阅读水平），其可能随着时间的推移以或多或少的影响来影响未来的事件。此外，学习环境的结构为每个学生创造了一个观察的视野，使一些事件（例如，另一组学生的对话）或多或少可见。最后，学生的不同特点（年龄、文化或种族背景、性别认同、是否以母语或非母语授课）可能导致他们以不同的方式对事件作出反应。现存的学习分析技术通过滞后来解释先前事件的影响：例如，使用一些固定数量的先前事件来预测未来事件。TMA将使这些相同的技术不依赖于事件本身的属性，而是依赖于代表不同学习模式之间相互作用的主张或假设的基本功能，学习环境的结构，以及学生作为STEM学习者可能系统性差异的方式。项目团队假设，TMA模型将为不同的学习者提供更细致、更准确、更公平的STEM学习过程视图。这种方法将扩大对有效的多模式STEM学习过程的理解，并允许研究人员考虑多样性并解决多模式STEM学习中的公平问题。TMA将首先作为一组算法进行开发和测试，用于使用三种广泛使用的学习分析工具进行跨模态分析：过程挖掘，认知网络分析和动态贝叶斯网络。研究人员的目标是使用模拟研究和对实际STEM学习数据集的分析来解决关于学习科学的两个基本研究问题：（1）在什么条件下（如果有的话），STEM学习过程的跨模态模型比单峰模型更能提供信息？以及（2）TMA能否为STEM学习者的少数群体建立跨模态学习过程中有意义的差异模型？该项目由NSF的EHR核心研究（ECR）计划支持。ECR计划强调基础STEM教育研究，产生该领域的基础知识。投资是在关键领域是必不可少的，广泛的和持久的：干学习和干学习环境，扩大参与干，干劳动力发展。该计划支持积累强有力的证据，为理解、建立理论解释、提出干预和创新建议以应对教育中持续存在的挑战提供信息。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In modern classrooms, students learn science, technology, engineering, and mathematics (STEM) through interactions not only with teachers and textbooks but also with computer games and simulations, automated tutors, and online resources. Education researchers thus have access to large amounts of data about students’ STEM learning processes, from classroom or online conversations to detailed records of student activity in educational apps. Despite the potential of such rich data for curriculum development and personalized assessment, there are significant technical and conceptual challenges to analyzing data that come from different sources or modalities. To address these challenges, this project will develop and test trans-modal analysis (TMA). TMA is a statistical technique and software package that will help researchers better and more easily integrate multiple types of data into analyses of STEM learning. This will enable more accurate understanding of students’ STEM learning processes, and in turn help identify potential inequities in assessment of student learning, informing education policy and practice for diverse learners. This project will include post-doctoral scholars, graduate student researchers, and undergraduate research interns, who will develop skills and experience in data science, learning analytics, software development, and scientific communication, providing training and mentoring for the next generation of education researchers. Although most analyses of learning processes are based on a single type or modality of data, STEM learning typically takes place in a multimodal setting. Models of STEM learning processes thus need to account for multiple sources and types of data to account for complex interactions between learners and the setting(s) in which they learn. For example, there are different types of events (questions from a teacher, chats with a peer, views of a resource) and different properties of events (gender of a person gesturing, linguistic fluency of a speaker, reading level of a person reading a document) that may influence future events with more or less impact over time. In addition, the structure of a learning environment creates a horizon of observation for each student, making some events (e.g., a conversation in another group of students) more or less visible. Finally, different characteristics of students (age, cultural or ethnic background, gender identification, whether instruction is in their native language or a non-native language) may lead them to respond to events in different ways. Extant learning analytic techniques account for the influence of prior events by lagging: for example, using some fixed number of prior events to predict future events. TMA will enable those same techniques to operate not on properties of the events themselves but on underlying functions that represent claims or hypotheses about the interaction between different learning modalities, the structure of the learning environment, and the ways in which students might systematically differ as STEM learners. The project team hypothesizes that TMA models will provide a more nuanced, more accurate, and more equitable view of STEM learning processes for diverse learners. This approach will expand the understanding of effective multi-modal STEM learning processes and allow researchers to account for diversity and address questions of equity in multi-modal STEM learning. TMA will be developed and tested first as a set of algorithms for conducting trans-modal analyses with three widely used learning analytic tools: process mining, epistemic network analysis, and dynamic Bayesian networks. The investigators aim to use simulation studies and the analysis of actual STEM learning datasets to address two fundamental research questions regarding the science of learning: (1) Under what conditions (if any) are trans-modal models of STEM learning processes more informative than uni-modal models? And (2) Can TMA model meaningful differences in trans-modal learning processes for minoritized groups of STEM learners?This project is supported by NSF's EHR Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development. The program supports the accumulation of robust evidence to inform efforts to understand, build theory to explain, and suggest intervention and innovations to address persistent challenges in education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.