CAREER: Complex Causal Moderated Mediation Analysis in Multisite Randomized Trials: Uncovering the Black Box Underlying the Impact of Educational Interventions on Math Performance

旨在提高数学成绩的干预措施经常在学校环境中实施。多点随机试验，或研究，其中个人被随机分配到学校的条件，经常被用来评估这些干预的效果。这些研究中最常见的问题是干预是否有效。尽管确定干预是否有效是一个重要的问题，但它并没有提供干预的全面图景。研究还应旨在确定干预如何发挥作用，或干预背后的机制，即所谓的调解机制。此外，该机制的效果可能会因学生特征、学校环境、当地实施情况或许多其他因素而有所不同，即所谓的缓和调解。评估有节制的调解对于了解干预对谁有效、在什么情况下有效以及为什么有效至关重要。目前，还不存在统计方法和工具来揭示多点试验中复杂的调解机制。这项研究的目的是开发方法和工具，使研究人员能够回答在多地点随机试验中对揭示复杂的中介机制至关重要的问题，包括(1)干预的总影响是如何由一个或多个同时或顺序的中介调节的，以及(2)这些机制如何随个体和背景因素的变化而变化。这些方法和工具将帮助研究人员更好地了解干预措施的复杂效果，以便他们能够针对不同的个人和学校背景改进和定制干预措施，从而提高教育公平。本项目将开发多地点因果调节调解分析的分析程序，其中包括一个调解人或多个并发或顺序调解人。因果关系的识别依赖于没有不可测量的混淆的假设，这在实际应用中通常是被违反的。因此，该项目还将开发直观的敏感性分析策略，以评估不仅未测量的前混杂和处理后混杂的潜在影响。这些方法将被应用于全国学习心态研究(NSLM)和启蒙影响研究(HSIS)，以调查教育干预对数学成绩及其异质性影响的中介机制。将进行全面的蒙特卡罗模拟来评估所提出的方法的性能。将开发一个具有图形界面的用户友好的R包，以使经验研究人员能够生成一套关于因果调解机制在个人和背景中的异质性的新的、全面、准确和有效的经验证据。该包的可用性将通过模拟、真实数据分析和焦点小组进行测试。将提供课程和讲习班，培训不同的学生和学者采用本项目开发的分析框架和工具。所有软件文件和培训材料都将向公众免费开放。这是一个学院早期职业发展计划项目，响应了国家科学基金会范围内的一项活动，该活动提供最负盛名的奖项，以支持有潜力在研究和教育领域充当学术榜样的早期职业教师。该项目得到了NSF的EDU核心研究(ECR)计划的支持。ECR计划强调基础STEM教育研究，以产生该领域的基础知识。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，认为值得支持。

Interventions aimed at improving mathematics performance are frequently implemented in school settings. Multisite randomized trials, or studies in which individuals are randomized to condition within schools, are frequently used to assess the effect of these interventions. The most common question in these studies is whether or not the intervention is effective. While determining whether an intervention is effective is an important question, it does not provide a comprehensive picture of the intervention. Studies should also be designed to determine how an intervention works, or the mechanisms underlying the intervention, known as the mediation mechanisms. In addition, the effect of the mechanism may differ depending on student characteristics, school context, local implementation, or a host of other factors, known as moderated mediation. Assessing moderated mediation is critical in understanding for whom and under what contexts the intervention is effective, and why. Currently, statistical methods and tools do not exist for unpacking complex mediation mechanisms in multisite trials. The purpose of this study is to develop methods and tools to enable researchers to answer questions crucial for unpacking complex mediation mechanisms in multisite randomized trials, including (1) how the total impact of an intervention is mediated by one mediator or multiple concurrent or sequential mediators and (2) how the mechanisms vary by individual and contextual factors. These methods and tools will help researchers better understand the complex effects of interventions so that they can improve and tailor interventions to different individuals and school contexts and thus enhance educational equity.This project will develop analysis procedures for multisite causal moderated mediation analysis with one mediator or multiple concurrent or sequential mediators. The identification of the causal effects relies on the assumption of no unmeasured confounding, which is usually violated in real applications. Therefore, this project will also develop intuitive sensitivity analysis strategies to assess the potential influence of not only unmeasured pretreatment confounding but also posttreatment confounding. The methods will be applied to the National Study of Learning Mindsets (NSLM) and Head Start Impact Study (HSIS) to investigate the mediation mechanisms underlying the impact of educational interventions on math performance and their heterogeneity. Comprehensive Monte Carlo simulations will be conducted to evaluate the performance of the proposed methods. A user-friendly R package with a graphical interface will be developed to enable empirical researchers to generate a new set of thorough, precise, and valid empirical evidence regarding the heterogeneity of causal mediation mechanisms across individuals and contexts. The usability of the package will be tested through simulations, real data analyses, and focus groups. Courses and workshops will be offered to train diverse students and scholars to adopt the analytic framework and tools developed from this project. All the software documentation and training materials will be publicly and freely accessible. This is a Faculty Early Career Development Program project responsive to a National Science Foundation-wide activity that offers the most prestigious awards in support of early-career faculty who have the potential to serve as academic role models in research and education. This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.