Improving Evaluations of R&D in STEM Education

这组研讨会的主要目标是为STEM教育研究人员提供他们需要的框架、技能和社区，以便将因果推理方法的新发展应用到他们的研究中。这些方法将立即在其当前(或不久的将来)研究中实施，并将产生更有力的因果关系发现，为新创新广泛改善STEM教育的潜力提供更高质量的证据。此外，第二个目标是为讲习班的研究生助理(统计学专业的学生)提供坚实的基础，了解当今STEM教育研究人员面临的现实问题。通过沉浸在这个社区中，目标是提高他们的沟通技能，同时也为他们提供机会开发新的方法，以解决当今STEM教育界面临的问题。STEM教育研究和发展研究往往侧重于开发和迭代完善旨在增加STEM参与度和技能的干预措施。由于大规模随机试验通常不太可能，研究人员通常使用相关方法来探索干预的效果。然而，在过去的几年里，统计学家开发了一系列广泛的方法来理解因果关系，这些方法不需要这些大规模的随机试验。虽然这些因果推理方法现在在医学和教育政策等领域很常见，但在STEM教育领域却很少见。这组研讨会的目的是向STEM教育研究人员介绍这些方法以及它们与他们已经使用的三个研究设计的关系：(1)在单一变量(例如，年龄、性别)上进行匹配，(2)测试前和测试后的比较，以及(3)实验室实验。除了介绍这些新的发展，还将包括关于混淆、有效性类型和权衡、设计敏感性、影响大小报告和有问题的研究实践(例如p-hack)的更广泛的讨论。该项目由三个EHR计划联合资助：IUSE计划(改进本科生STEM教育计划)；DRK-12计划(发现研究PREK-12计划)；ECR(教育核心研究计划)。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The primary goal of this set of workshops is to provide STEM education researchers with the framework, skills, and community they need to implement new developments in causal inference methods into their research. These methods will be immediately implementable in their current (or near future) studies and will result in stronger causal findings, providing higher-quality evidence regarding the potential of new innovations to improve STEM education broadly. Additionally, a secondary goal is to provide the graduate assistants at the workshop (students in statistics) with a strong foundation in the real-world problems facing researchers in STEM education today. By being immersed in this community, the goal is to improve their communication skills, while also providing them with opportunities to develop new methods that address problems facing the STEM education community today.STEM education research and development studies often focus on the development and iterative refinement of interventions meant to increase STEM participation and skills. Since large-scale randomized experiments are not often possible, researchers typically use correlational methods instead to explore the effects of interventions. Over the past several years, however, statisticians have developed a broad array of methods for understanding causality that do not require these large-scale randomized trials. While these causal inference methods are now common in fields like medicine and education policy, they are much less commonly found in STEM education fields. The purpose of this set of workshops is to introduce STEM education researchers to these methods and how they relate to three research designs they already use: (1) matching on a single variable (e.g., age, gender), (2) pre-test post-test comparisons, and (3) lab experiments. In addition to introducing these new developments, broader discussions of confounding, validity types and trade-offs, design sensitivity, effect size reporting, and questionable research practices (e.g., p-hacking) will also be included.This project is jointly funded by three EHR programs: the IUSE program (Improving Undergraduate STEM Education program); the DRK-12 program (Discovery Research preK-12 program); and ECR (Education Core Research program).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.