CAREER: Complex Causal Moderated Mediation Analysis in Multisite Randomized Trials: Uncovering the Black Box Underlying the Impact of Educational Interventions on Math Performance

旨在提高数学成绩的干预措施经常在学校实施。多地点随机试验，或将个体随机分配到学校内的研究，经常用于评估这些干预措施的效果。这些研究中最常见的问题是干预是否有效。虽然确定干预措施是否有效是一个重要的问题，但它并不能提供干预措施的全面情况。还应设计研究以确定干预措施如何起作用，或干预措施的基础机制，即调解机制。此外，该机制的效果可能因学生特征、学校背景、当地实施或许多其他因素而异，这些因素被称为有调节的中介。评估适度调解对于理解干预对谁、在什么情况下有效以及为什么有效至关重要。目前，统计方法和工具不存在，以解开复杂的中介机制在多地点试验。本研究的目的是开发方法和工具，使研究人员能够回答在多地点随机试验中解开复杂中介机制的关键问题，包括：(1)干预的总影响如何由一个中介或多个并发或顺序中介介导；(2)机制如何因个体和环境因素而变化。这些方法和工具将帮助研究人员更好地理解干预措施的复杂效果，以便他们能够根据不同的个人和学校环境改进和定制干预措施，从而提高教育公平。本项目将开发一个或多个并发或顺序中介的多站点因果调节中介分析的分析程序。因果效应的识别依赖于没有未测量混淆的假设，这在实际应用中通常是违反的。因此，本项目还将开发直观的敏感性分析策略，以评估未测量的预处理混杂和后处理混杂的潜在影响。该方法将应用于国家学习心态研究（NSLM）和学前教育影响研究（HSIS），探讨教育干预对数学成绩影响的中介机制及其异质性。将进行全面的蒙特卡罗模拟来评估所提出方法的性能。将开发一个具有图形界面的用户友好的R包，以使实证研究人员能够生成一套关于个体和环境之间因果中介机制异质性的新的全面，精确和有效的实证证据。该软件包的可用性将通过模拟、真实数据分析和焦点小组进行测试。将提供课程和讲习班，培训不同的学生和学者采用本项目开发的分析框架和工具。所有软件文档和培训材料都将公开和免费提供。这是一个教师早期职业发展计划项目，响应国家科学基金会范围内的活动，提供最负盛名的奖项，以支持有潜力在研究和教育中成为学术榜样的早期职业教师。本项目由美国国家科学基金会EDU核心研究（ECR）项目支持。ECR项目强调在该领域产生基础知识的基础STEM教育研究。投资在至关重要、广泛和持久的关键领域：STEM学习和STEM学习环境，扩大STEM参与，以及STEM劳动力发展。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Interventions aimed at improving mathematics performance are frequently implemented in school settings. Multisite randomized trials, or studies in which individuals are randomized to condition within schools, are frequently used to assess the effect of these interventions. The most common question in these studies is whether or not the intervention is effective. While determining whether an intervention is effective is an important question, it does not provide a comprehensive picture of the intervention. Studies should also be designed to determine how an intervention works, or the mechanisms underlying the intervention, known as the mediation mechanisms. In addition, the effect of the mechanism may differ depending on student characteristics, school context, local implementation, or a host of other factors, known as moderated mediation. Assessing moderated mediation is critical in understanding for whom and under what contexts the intervention is effective, and why. Currently, statistical methods and tools do not exist for unpacking complex mediation mechanisms in multisite trials. The purpose of this study is to develop methods and tools to enable researchers to answer questions crucial for unpacking complex mediation mechanisms in multisite randomized trials, including (1) how the total impact of an intervention is mediated by one mediator or multiple concurrent or sequential mediators and (2) how the mechanisms vary by individual and contextual factors. These methods and tools will help researchers better understand the complex effects of interventions so that they can improve and tailor interventions to different individuals and school contexts and thus enhance educational equity.This project will develop analysis procedures for multisite causal moderated mediation analysis with one mediator or multiple concurrent or sequential mediators. The identification of the causal effects relies on the assumption of no unmeasured confounding, which is usually violated in real applications. Therefore, this project will also develop intuitive sensitivity analysis strategies to assess the potential influence of not only unmeasured pretreatment confounding but also posttreatment confounding. The methods will be applied to the National Study of Learning Mindsets (NSLM) and Head Start Impact Study (HSIS) to investigate the mediation mechanisms underlying the impact of educational interventions on math performance and their heterogeneity. Comprehensive Monte Carlo simulations will be conducted to evaluate the performance of the proposed methods. A user-friendly R package with a graphical interface will be developed to enable empirical researchers to generate a new set of thorough, precise, and valid empirical evidence regarding the heterogeneity of causal mediation mechanisms across individuals and contexts. The usability of the package will be tested through simulations, real data analyses, and focus groups. Courses and workshops will be offered to train diverse students and scholars to adopt the analytic framework and tools developed from this project. All the software documentation and training materials will be publicly and freely accessible. This is a Faculty Early Career Development Program project responsive to a National Science Foundation-wide activity that offers the most prestigious awards in support of early-career faculty who have the potential to serve as academic role models in research and education. This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.