Assessing Complex Collaborative STEM Learning at Scale with Epistemic Network Analysis

通过认知网络分析大规模评估复杂的协作 STEM 学习

• 批准号: 1661036
• 负责人: David Shaffer
• 金额: $ 249.98万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1661036&HistoricalAwards=false
• 关键词: Assessing; Complex; Collaborative; STEM; Learning

This project was submitted in response to EHR Core Research (ECR) program announcement NSF 15-509. The ECR program of fundamental research in STEM education provides funding in critical research areas that are essential, broad and enduring. EHR seeks proposals that will help synthesize, build and/or expand research foundations in the following focal areas: STEM learning, STEM learning environments, STEM workforce development, and broadening participation in STEM. The ECR program is distinguished by its emphasis on the accumulation of robust evidence to inform efforts to (a) understand, (b) build theory to explain, and (c) suggest interventions (and innovations) to address persistent challenges in STEM interest, education, learning, and participation.In this project, researchers will develop a statistical analysis technique for measuring how people learn. With prior NSF funding, members of the team created epistemic network analysis (ENA), a technique for creating network models of complex and collaborative thinking in science, technology, engineering, and mathematics. ENA is being used by more than 40 researchers at 19 universities to answer a wide range of research questions in learning sciences, cognitive neuroscience, engineering education, environmental science education, medical and surgical education, and history of science. The proposed research and development will create an online toolkit that lets researchers upload audio, video, text, or log-file data, automatically transcribe the audio data, develop and validate automated codes using supervised natural language processing tools, and produce ENA models. This will make it possible for researchers analyze data on how people learn without requiring simultaneous expertise in automated transcription, data segmentation, coding, and network modeling. It will also make it possible to conduct analyses of learning using the large volumes of data that are currently generated by online learning tools, significantly expanding capacity for research on learning.In this project, the research team will conduct fundamental research on learning in science, technology, engineering, and mathematics (STEM) by developing and extending a theory-based statistical analysis technique for using network analysis to model complex and collaborative STEM thinking (CCST). With prior NSF funding, members of the team created epistemic network analysis (ENA), a technique for creating dynamic models of CCST. Data on CCST typically come in one of two forms: Video and audio recordings of interactions from classrooms and workplaces, or log files from online interactions. Data in audio or video form must be transcribed. Text data from recordings or log files have to be coded, or annotated to indicate what elements of CCST are present in the data, and where those elements are located. With existing tools, researchers must complete these steps by hand, or use their own techniques for automated transcription and coding. However, many skilled CCST researchers are not simultaneously experts in the sciences of automated transcription, automated identification, validation, and application of codes, and network analysis. The proposed research and development will create ENAlysis, an online toolkit that provides a seamless, automated analysis pipeline from raw data to final results. The project will thus develop innovative methods for measuring STEM learning and expand access to a proven, theory-based technique for analyzing CCST. The result will be expanded use of powerful learning analytic techniques to model CCST, which will significantly improve assessment of STEM thinking and learning and inform policy and practice.

该项目是为了响应EHR核心研究（ECR）计划公告NSF 15-509而提交的。在STEM教育的基础研究ECR计划提供资金在关键的研究领域是必不可少的，广泛的和持久的。EHR寻求有助于综合，建立和/或扩大以下重点领域研究基础的建议：STEM学习，STEM学习环境，STEM劳动力发展和扩大STEM参与。ECR项目的特点是强调积累强有力的证据，为以下努力提供信息：（a）理解，（B）建立理论解释，（c）提出干预措施（和创新），以解决STEM兴趣，教育，学习和参与方面的持续挑战。在该项目中，研究人员将开发一种统计分析技术，用于测量人们如何学习。在之前的NSF资助下，团队成员创建了认知网络分析（ENA），这是一种用于创建科学，技术，工程和数学中复杂和协作思维的网络模型的技术。19所大学的40多名研究人员正在使用ENA来回答学习科学、认知神经科学、工程教育、环境科学教育、医学和外科教育以及科学史等领域的广泛研究问题。拟议的研究和开发将创建一个在线工具包，让研究人员上传音频，视频，文本或日志文件数据，自动转录音频数据，使用监督自然语言处理工具开发和验证自动代码，并生成ENA模型。这将使研究人员能够分析人们如何学习的数据，而不需要同时具备自动转录、数据分割、编码和网络建模方面的专业知识。该项目还将利用在线学习工具目前产生的大量数据对学习进行分析，大大扩大学习研究的能力。在该项目中，研究小组将在科学、技术、工程、教育和科学领域进行学习的基础研究。通过开发和扩展基于理论的统计分析技术，使用网络分析对复杂和协作的STEM思维（CCST）进行建模，在先前的NSF资助下，该团队的成员创建了认知网络分析（ENA），这是一种用于创建CCST动态模型的技术。CCST上的数据通常以两种形式之一出现：教室和工作场所交互的视频和音频记录，或在线交互的日志文件。音频或视频形式的数据必须转录。来自记录或日志文件的文本数据必须进行编码或注释，以指示数据中存在CCST的哪些元素以及这些元素的位置。使用现有的工具，研究人员必须手工完成这些步骤，或者使用他们自己的技术进行自动转录和编码。然而，许多熟练的CCST研究人员并不同时是自动转录，自动识别，验证和代码应用以及网络分析科学的专家。拟议的研究和开发将创建ENAlysis，这是一个在线工具包，提供从原始数据到最终结果的无缝自动分析管道。因此，该项目将开发衡量STEM学习的创新方法，并扩大获得经过验证的基于理论的技术来分析CCST。其结果将扩大使用强大的学习分析技术来模拟CCST，这将显着改善对STEM思维和学习的评估，并为政策和实践提供信息。

项目成果

Quality and Safety Education for Nursing (QSEN) in Virtual Reality Simulations: A Quantitative Ethnographic Examination

虚拟现实模拟中的护理质量与安全教育 (QSEN)：定量人种学检查

• DOI: 10.1007/978-3-030-93859-8_16
• 发表时间: 2022
• 期刊: International Conference on Quantitative Ethnography 2021
• 作者: Shah, M.;Siebert-Evenstone, A.;Moots, H.;Eagan, B.
• 通讯作者: Eagan, B.

Testing the reliability of inter-rater reliability

• DOI: 10.1145/3375462.3375508
• 发表时间: 2020-03
• 期刊: Proceedings of the Tenth International Conference on Learning Analytics & Knowledge
• 作者: Brendan R. Eagan;Jais Brohinsky;Jingyi Wang;D. Shaffer

Big Data for Thick Description of Deep Learning

大数据深度学习深度描述

• 发表时间: 2018
• 期刊: Deep learning: Multi-disciplinary approaches
• 作者: Shaffer, D.

SENS: Network analytics to combine social and cognitive perspectives of collaborative learning

• DOI: 10.1016/j.chb.2018.07.003
• 发表时间: 2019-03-01
• 期刊: COMPUTERS IN HUMAN BEHAVIOR
• 影响因子: 9.9
• 作者: Gasevic, Dragan;Joksimovic, Srecko;Shaffer, David Williamson
• 通讯作者: Shaffer, David Williamson

Dissecting learning tactics in MOOC using ordered network analysis

• DOI: 10.1111/jcal.12735
• 发表时间: 2022-08-30
• 期刊: JOURNAL OF COMPUTER ASSISTED LEARNING
• 影响因子: 5
• 作者: Fan, Yizhou;Tan, Yuanru;Gasevic, Dragan
• 通讯作者: Gasevic, Dragan