Meta-Analysis of Effectiveness of Simulation and Adaptive Learning Systems in STEM Education

该提案是根据EHR核心研究（ECR）项目公告NSF 15-509提交的。STEM教育基础研究ECR项目为重要、广泛和持久的关键研究领域提供资金。EHR寻求有助于在以下重点领域综合、建立和/或扩大研究基础的提案：STEM学习、STEM学习环境、STEM劳动力发展和扩大STEM参与。ECR项目的特点是强调积累有力的证据，为以下方面的努力提供信息：(a)理解，(b)建立理论来解释，(c)提出干预（和创新）措施，以解决STEM兴趣、教育、学习和参与方面的持续挑战。模拟和自适应学习系统广泛应用于从K-12到研究生水平的STEM教育中。它们的使用得到了美国教育部2016年国家技术计划的认可。利用模拟来提供丰富的学习经验和及时的形成性评估，与以能力为基础的教育趋势和美国国家科学基金会在推进网络学习方面的工作是一致的。然而，缺乏利用这两种技术吸引学习者并提高学习成果的循证实践。为了解决这一迫切需求，该项目将综合有关STEM教育中两种技术有效性的现有知识，并确定基于证据的技术使用实践。这些努力将产生有关如何有效利用技术改造STEM教育的新知识，为美国建立一支更大、更强的STEM劳动力队伍，并最终确保国家未来的繁荣、创新潜力和全球经济竞争力。项目团队将进行两项独立的元分析，以量化模拟和适应性学习系统对STEM学习和参与的影响。荟萃分析是一种强大的方法，可以综合多个研究的结果，并产生可靠的累积证据，为未来的研究和实践提供信息。该项目将侧重于评估模拟和适应性学习系统在STEM教育中的有效性的研究。对于包含在meta分析中的每项研究，将计算技术使用对学习者的参与（例如，参与，享受，兴趣）和成就（例如，成绩）的影响大小。效应量是一种标准化的统计度量，用于量化效应的大小，并允许对不同研究的效应进行比较。将汇总所有纳入研究的效应量，以估计每种技术的总体效应量以及效应量在不同研究之间的差异程度。还将进行分析，以确定效应量与技术使用方式（即教学法）的关系程度，以及效应量是否因学习者特征（例如，性别、年龄、种族）和学习环境（例如，交付模式、课程、设置、机构）而变化。该项目将为每种技术的有效性提供及时有力的证据，为STEM学习、教学、未来研究和技术开发提供信息

This proposal was submitted in response to EHR Core Research (ECR) program announcement NSF 15-509. The ECR program of fundamental research in STEM education provides funding in critical research areas that are essential, broad and enduring. EHR seeks proposals that will help synthesize, build and/or expand research foundations in the following focal areas: STEM learning, STEM learning environments, STEM workforce development, and broadening participation in STEM. The ECR program is distinguished by its emphasis on the accumulation of robust evidence to inform efforts to (a) understand, (b) build theory to explain, and (c) suggest interventions (and innovations) to address persistent challenges in STEM interest, education, learning, and participation.Simulation and adaptive learning systems are widely used instructional technologies in STEM education from K-12 to graduate levels. Their use was endorsed by the U.S. Department of Education in its 2016 national technology plan. Using simulation to provide rich learning experiences and timely formative assessment is aligned with the trend towards competency-based education and the NSF's work in advancing cyberlearning. However, evidence-based practices for using the two technologies to engage learners and enhance learning outcomes are lacking. To address this urgent need, this project will synthesize existing knowledge on the effectiveness of two technologies in STEM education and identify evidence-based practices of technology use. These efforts will generate new knowledge about how to use technology effectively to transform STEM education, build a larger and stronger STEM workforce for the U.S., and ultimately ensure the nation's future prosperity, innovation potential, and global economic competitiveness. The project team will conduct two separate meta-analyses to quantify the effects of simulation and adaptive learning systems on STEM learning and engagement. Meta-analysis is a powerful method to synthesize findings from multiple research studies and produce robust and accumulative evidence to inform future research and practice. This project will focus on research studies that evaluated the effectiveness of simulation and adaptive learning systems in STEM education. For each study included in meta-analyses, effect sizes of technology use on learners' engagement (e.g., participation, enjoyment, interest) and achievement (e.g., grades) will be calculated. Effect size is a standardized statistical measure that quantifies the magnitude of effects and allows comparison of effects across research studies. Effect sizes from all included studies will be pooled to estimate an overall effect size for each technology and the degree to which effect sizes vary from one study to another. Analyses will also be conducted to determine the extent to which effect sizes are related to how technology was used (i.e., pedagogy) and whether effect sizes vary by learner characteristics (e.g., gender, age, ethnicity) and learning environments (e.g., delivery modes, courses, settings, institutions). This project will provide timely and robust evidence regarding the effectiveness of each technology to inform STEM learning, teaching, future research, and technology development