CAREER: Complex Causal Moderated Mediation Analysis in Multisite Randomized Trials: Uncovering the Black Box Underlying the Impact of Educational Interventions on Math Performance

职业：多地点随机试验中的复杂因果调节中介分析：揭示教育干预对数学成绩影响的黑匣子

• 批准号: 2337612
• 负责人: Xu Qin
• 金额: $ 84.25万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-06-01 至 2029-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2337612&HistoricalAwards=false
• 关键词: CAREER; Complex; Causal; Moderated; Mediation

Interventions aimed at improving mathematics performance are frequently implemented in school settings. Multisite randomized trials, or studies in which individuals are randomized to condition within schools, are frequently used to assess the effect of these interventions. The most common question in these studies is whether or not the intervention is effective. While determining whether an intervention is effective is an important question, it does not provide a comprehensive picture of the intervention. Studies should also be designed to determine how an intervention works, or the mechanisms underlying the intervention, known as the mediation mechanisms. In addition, the effect of the mechanism may differ depending on student characteristics, school context, local implementation, or a host of other factors, known as moderated mediation. Assessing moderated mediation is critical in understanding for whom and under what contexts the intervention is effective, and why. Currently, statistical methods and tools do not exist for unpacking complex mediation mechanisms in multisite trials. The purpose of this study is to develop methods and tools to enable researchers to answer questions crucial for unpacking complex mediation mechanisms in multisite randomized trials, including (1) how the total impact of an intervention is mediated by one mediator or multiple concurrent or sequential mediators and (2) how the mechanisms vary by individual and contextual factors. These methods and tools will help researchers better understand the complex effects of interventions so that they can improve and tailor interventions to different individuals and school contexts and thus enhance educational equity.This project will develop analysis procedures for multisite causal moderated mediation analysis with one mediator or multiple concurrent or sequential mediators. The identification of the causal effects relies on the assumption of no unmeasured confounding, which is usually violated in real applications. Therefore, this project will also develop intuitive sensitivity analysis strategies to assess the potential influence of not only unmeasured pretreatment confounding but also posttreatment confounding. The methods will be applied to the National Study of Learning Mindsets (NSLM) and Head Start Impact Study (HSIS) to investigate the mediation mechanisms underlying the impact of educational interventions on math performance and their heterogeneity. Comprehensive Monte Carlo simulations will be conducted to evaluate the performance of the proposed methods. A user-friendly R package with a graphical interface will be developed to enable empirical researchers to generate a new set of thorough, precise, and valid empirical evidence regarding the heterogeneity of causal mediation mechanisms across individuals and contexts. The usability of the package will be tested through simulations, real data analyses, and focus groups. Courses and workshops will be offered to train diverse students and scholars to adopt the analytic framework and tools developed from this project. All the software documentation and training materials will be publicly and freely accessible. This is a Faculty Early Career Development Program project responsive to a National Science Foundation-wide activity that offers the most prestigious awards in support of early-career faculty who have the potential to serve as academic role models in research and education. This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在学校环境中经常实施旨在提高数学成绩的干预措施。多地点随机试验，或在学校内将个人随机分配到不同条件的研究，经常被用来评估这些干预措施的效果。这些研究中最常见的问题是干预是否有效。虽然确定一项干预措施是否有效是一个重要问题，但它并不能全面反映干预措施的情况。还应设计研究，以确定干预措施如何发挥作用，或干预措施背后的机制，即所谓的调解机制。此外，该机制的效果可能因学生特点、学校环境、当地实施或其他因素而异，称为适度调解。评估有节制的调解对于了解干预对谁和在什么情况下有效以及为什么有效至关重要。目前，还没有统计方法和工具来分析多地点审判中复杂的调解机制。本研究的目的是开发方法和工具，使研究人员能够回答在多地点随机试验中解开复杂调解机制的关键问题，包括（1）干预的总影响如何由一个调解人或多个并发或顺序调解人调解，以及（2）机制如何因个体和背景因素而异。这些方法和工具将帮助研究人员更好地了解干预措施的复杂效果，使他们能够改进和调整干预措施，以不同的个人和学校的情况，从而提高教育公平。本项目将开发多站点因果调节调解分析与一个调停人或多个并发或顺序mediators的分析程序。因果效应的识别依赖于没有不可测量的混杂的假设，这在真实的应用中通常是违反的。因此，该项目还将开发直观的敏感性分析策略，以评估不仅未测量的治疗前混杂因素，而且治疗后混杂因素的潜在影响。该方法将被应用到学习心态的国家研究（NSLM）和头开始影响研究（HSIS）调查的中介机制的影响，教育干预措施对数学成绩及其异质性。将进行全面的蒙特卡罗模拟，以评估所提出的方法的性能。一个用户友好的R包与图形界面将开发，使实证研究人员产生一套新的彻底的，精确的，有效的实证证据的异质性的因果中介机制在个人和背景。该软件包的可用性将通过模拟、真实的数据分析和焦点小组进行测试。将提供课程和讲习班，以培训不同的学生和学者采用该项目开发的分析框架和工具。所有软件文档和培训材料都将公开和免费提供。这是一个教师早期职业发展计划项目，响应国家科学基金会范围内的活动，提供最负盛名的奖项，以支持有潜力在研究和教育中担任学术榜样的早期职业教师。该项目由NSF的EDU核心研究（ECR）计划支持。ECR计划强调基础STEM教育研究，产生该领域的基础知识。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响力审查标准进行评估，被认为值得支持。