Improving school governance and learning outcomes at scale: A randomized evaluation of the Madhya Pradesh School Quality Assessment program

我们建议进行一项随机对照试验（RCT），研究发展中国家政府最近最雄心勃勃的尝试之一，以大规模改善教育治理。评估将特别关注农村部落地区的学校治理状况、流程和学习成果，这一直是改善印度教育最具挑战性的背景之一。中央邦（MP）学校质量评估（MPSQA）干预措施是一项综合方案，旨在通过以下措施改善学校治理和成果：（a）定期监测学校，（B）制作学校成绩单和定制的学校改进计划，（c）每季度跟踪该计划的进展情况，（d）利用信息和通信技术工具收集和报告所有评估结果、学校改进计划和后续行动的实时记录，并提交给专门的在线系统。PI与项目实施团队合作，并在2015/16学年成功地在约20，000所学校随机推出该计划。政府目前希望到2019年将该计划普及到MP的所有12万所学校。评估的时间是理想的：发现积极的结果可能有助于加快在MP和印度其他地方的推广，虽然没有发现显著的积极结果，（沿着中间因素的数据，以了解无影响的原因）可能会促使重新评估在将资源用于可能不会对过程或结果产生预期影响的规模扩大之前，对计划及其实施进行评估。评估将评估该计划对关键过程指标（如教师缺席，努力和教学法）以及学生两年的学术成果的影响。拟议的评估将集中在2015/16年度全州范围内随机推出的计划，并寻求回答三个核心问题：（a）如何学校一级的过程，投入和学习成果在公共教育系统内的不同偏远部落地区和国家的其他部分？(b)MPSQA干预措施在多大程度上改善了计划学校的学校流程和学习成果，偏远部落地区和该州其他地区的学校之间是否存在差异;以及（c）实现和抑制计划效果的途径是什么（例如，教师努力的改善、更好的学校管理、更有重点的教学实践以及学生出勤率和努力）？在回答这些问题时，我们将寻求使用广泛的行政数据，包括全州范围内的低风险评估结果，以及独立管理的测试和通过定期突击学校访问随机选择的小学和中学学校子集收集的详细过程指标。这些独立管理的测试将防止项目学校操纵官方测试结果的可能性，并通过精心的测试设计将测试结果与印度和国际上的其他评估联系起来。核心提案和研究设计是基于使用调查数据的量化影响评估方法。然而，由于认识到全系统的大型改革往往有意想不到的方面，而且不同行为者和不同治理级别之间有多层次的互动，我们还提出了一个定性部分，力求对以下方面有一个补充性的、细致入微的理解：（a）从教育系统相关行为者的互动、看法和动机的角度来看，干预行动的背景（学生、教师、校长和教育公务员）（B）方案的效果如何通过教育系统的各个层面来调节，并了解这种改革的“有利因素”或瓶颈可能是什么。

We propose to conduct a randomized controlled trial (RCT) of one of the most ambitious recent attempts by a developing country government to improve education governance at scale. The evaluation will have a particular focus on the state of school governance, processes and learning outcomes in rural tribal areas, which have been one of the most challenging contexts to improve education in India.The Madhya Pradesh (MP) School Quality Assessment (MPSQA) intervention is a comprehensive program that aims to improve school governance and outcomes through a combination of (a) regular monitoring of schools, (b) creation of school report cards, and a customized school improvement plan, (c) quarterly follow-ups on progress against this plan, and (d) leveraging of ICT tools to collect and report real-time records of all assessment results, school improvement plans, and follow-ups to a dedicated online system. The PI's have worked with the project implementation team and have successfully randomized the rollout of the program across ~20,000 schools in the 2015/16 school year. The government is currently hoping to universalize the program to cover all 120,000 schools in MP by 2019. The timing of the evaluation is ideal: finding positive results may help accelerate the scaling up across MP and elsewhere in India, while not finding significant positive results (along with data on intermediate factors to understand reasons for non-impact) may prompt a re-evaluation of the program and its implementation before spending resources on a scale up that may not have the desired impacts on processes or outcomes.The evaluation will assess the effects of the program on key process metrics (e.g. teacher absence, effort and pedagogy) as well as student academic outcomes for two years. The proposed evaluation will focus on the randomized state-wide rollout of the program in 2015/16 and seek to answer three core questions: (a) How do school-level processes, inputs, and learning outcomes differ within the public education system between remote tribal areas and other parts of the state?; (b) To what extent does the MPSQA intervention improve school-level processes and learning outcomes in the program schools and does this differ between schools in remote tribal areas and other parts of the state; and (c) What are the pathways through which program effects are realized and inhibited (examples include improvements in teacher effort, in better school management, in better-focused pedagogical practices, and student attendance and effort)? In answering these questions, we will seek to use extensive administrative data including state-wide low-stakes assessment results, alongside independently-administered tests and detailed process metrics collected through regular unannounced school visits to a randomly-selected subset of schools at both elementary and secondary levels. These independently-administered tests will guard against the possibility of manipulation of official test results in program schools and also allow for the linking of test results, through careful test design, to other assessments in India and internationally. The core proposal and the study design are based on the use of quantitative impact evaluation methods using survey data. However, recognizing that large system-wide reforms often have unanticipated facets and multiple layers of interaction between different agents and different levels of governance, we also propose a qualitative component which seeks a complementary nuanced understanding of (a) the context in which the intervention operates, in terms of the interactions, perceptions and motivation of the relevant agents in the education system (students, teachers, headmasters and education civil servants) (b) how the program effects are mediated through these various layers of the education system and learning about what the 'enabling factors' or bottlenecks of such reforms might be.