Improving Evaluations of R&D in STEM Education

这一系列研讨会的主要目标是为STEM教育研究人员提供他们需要的框架，技能和社区，以便在他们的研究中实施因果推理方法的新发展。这些方法将立即在他们目前（或不久的将来）的研究中实施，并将产生更强的因果发现，提供更高质量的证据，证明新创新的潜力，以广泛改善STEM教育。此外，次要目标是为研讨会的研究生助理（统计学专业的学生）提供当今STEM教育研究人员面临的现实问题的坚实基础。通过沉浸在这个社区中，目标是提高他们的沟通技能，同时也为他们提供机会，开发新的方法，解决当今STEM教育社区面临的问题。STEM教育研究和发展研究通常侧重于开发和迭代改进干预措施，旨在提高STEM参与和技能。由于大规模的随机实验往往是不可能的，研究人员通常使用相关方法来探索干预措施的效果。然而，在过去的几年里，统计学家已经开发出了一系列广泛的方法来理解因果关系，而不需要这些大规模的随机试验。虽然这些因果推理方法现在在医学和教育政策等领域很常见，但在STEM教育领域却不太常见。这组研讨会的目的是向STEM教育研究人员介绍这些方法，以及它们如何与他们已经使用的三种研究设计相关联：（1）在单个变量上进行匹配（例如，年龄，性别），（2）前测后测比较，和（3）实验室实验。除了介绍这些新的发展，更广泛的讨论混杂，有效性类型和权衡，设计敏感性，效应量报告，和可疑的研究实践（例如，该项目由三个EHR项目共同资助：IUSE项目（改善本科STEM教育项目）、DRK-12项目（发现研究preK-12项目）和ECR（教育核心研究项目）。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The primary goal of this set of workshops is to provide STEM education researchers with the framework, skills, and community they need to implement new developments in causal inference methods into their research. These methods will be immediately implementable in their current (or near future) studies and will result in stronger causal findings, providing higher-quality evidence regarding the potential of new innovations to improve STEM education broadly. Additionally, a secondary goal is to provide the graduate assistants at the workshop (students in statistics) with a strong foundation in the real-world problems facing researchers in STEM education today. By being immersed in this community, the goal is to improve their communication skills, while also providing them with opportunities to develop new methods that address problems facing the STEM education community today.STEM education research and development studies often focus on the development and iterative refinement of interventions meant to increase STEM participation and skills. Since large-scale randomized experiments are not often possible, researchers typically use correlational methods instead to explore the effects of interventions. Over the past several years, however, statisticians have developed a broad array of methods for understanding causality that do not require these large-scale randomized trials. While these causal inference methods are now common in fields like medicine and education policy, they are much less commonly found in STEM education fields. The purpose of this set of workshops is to introduce STEM education researchers to these methods and how they relate to three research designs they already use: (1) matching on a single variable (e.g., age, gender), (2) pre-test post-test comparisons, and (3) lab experiments. In addition to introducing these new developments, broader discussions of confounding, validity types and trade-offs, design sensitivity, effect size reporting, and questionable research practices (e.g., p-hacking) will also be included.This project is jointly funded by three EHR programs: the IUSE program (Improving Undergraduate STEM Education program); the DRK-12 program (Discovery Research preK-12 program); and ECR (Education Core Research program).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.