Improving school governance and learning outcomes at scale: A randomized evaluation of the Madhya Pradesh School Quality Assessment program

大规模改善学校治理和学习成果：中央邦学校质量评估计划的随机评估

• 批准号: ES/N010388/1
• 负责人: Karthik Muralidharan
• 金额: $ 72.09万
• 依托单位: University of Madras
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2016
• 资助国家: 英国
• 起止时间: 2016 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=ES%2FN010388%2F1
• 关键词: Improving; governance; learning; outcomes; scale

We propose to conduct a randomized controlled trial (RCT) of one of the most ambitious recent attempts by a developing country government to improve education governance at scale. The evaluation will have a particular focus on the state of school governance, processes and learning outcomes in rural tribal areas, which have been one of the most challenging contexts to improve education in India.The Madhya Pradesh (MP) School Quality Assessment (MPSQA) intervention is a comprehensive program that aims to improve school governance and outcomes through a combination of (a) regular monitoring of schools, (b) creation of school report cards, and a customized school improvement plan, (c) quarterly follow-ups on progress against this plan, and (d) leveraging of ICT tools to collect and report real-time records of all assessment results, school improvement plans, and follow-ups to a dedicated online system. The PI's have worked with the project implementation team and have successfully randomized the rollout of the program across ~20,000 schools in the 2015/16 school year. The government is currently hoping to universalize the program to cover all 120,000 schools in MP by 2019. The timing of the evaluation is ideal: finding positive results may help accelerate the scaling up across MP and elsewhere in India, while not finding significant positive results (along with data on intermediate factors to understand reasons for non-impact) may prompt a re-evaluation of the program and its implementation before spending resources on a scale up that may not have the desired impacts on processes or outcomes.The evaluation will assess the effects of the program on key process metrics (e.g. teacher absence, effort and pedagogy) as well as student academic outcomes for two years. The proposed evaluation will focus on the randomized state-wide rollout of the program in 2015/16 and seek to answer three core questions: (a) How do school-level processes, inputs, and learning outcomes differ within the public education system between remote tribal areas and other parts of the state?; (b) To what extent does the MPSQA intervention improve school-level processes and learning outcomes in the program schools and does this differ between schools in remote tribal areas and other parts of the state; and (c) What are the pathways through which program effects are realized and inhibited (examples include improvements in teacher effort, in better school management, in better-focused pedagogical practices, and student attendance and effort)? In answering these questions, we will seek to use extensive administrative data including state-wide low-stakes assessment results, alongside independently-administered tests and detailed process metrics collected through regular unannounced school visits to a randomly-selected subset of schools at both elementary and secondary levels. These independently-administered tests will guard against the possibility of manipulation of official test results in program schools and also allow for the linking of test results, through careful test design, to other assessments in India and internationally. The core proposal and the study design are based on the use of quantitative impact evaluation methods using survey data. However, recognizing that large system-wide reforms often have unanticipated facets and multiple layers of interaction between different agents and different levels of governance, we also propose a qualitative component which seeks a complementary nuanced understanding of (a) the context in which the intervention operates, in terms of the interactions, perceptions and motivation of the relevant agents in the education system (students, teachers, headmasters and education civil servants) (b) how the program effects are mediated through these various layers of the education system and learning about what the 'enabling factors' or bottlenecks of such reforms might be.

我们建议对发展中国家政府最近最雄心勃勃的大规模改善教育治理的尝试之一进行随机对照试验（RCT）。评估将特别关注农村部落地区的学校治理状况、流程和学习成果，这些地区一直是印度改善教育最具挑战性的环境之一。中央邦（MP）学校质量评估（MPSQA）干预措施是一项综合性项目，旨在通过以下方式改善学校治理和成果：(a)定期监测学校；(b)创建学校成绩单和定制学校改进计划；(c)按季度跟进该计划的进展情况；(d)利用信息通信技术工具收集和报告所有评估结果、学校改进计划的实时记录；还有一个专门的在线系统。项目负责人与项目实施团队合作，在2015/16学年成功地在约2万所学校随机推出了该项目。政府目前希望到2019年将该计划普及到中央邦所有12万所学校。评估的时机是理想的：发现积极的结果可能有助于加速在MP和印度其他地方的规模扩大，而没有发现显著的积极结果（以及关于中间因素的数据，以了解无影响的原因）可能会促使重新评估项目及其实施，然后再花费资源进行规模扩大，这可能不会对过程或结果产生预期的影响。评估将评估该计划在关键过程指标（如教师缺勤、努力和教学法）以及学生两年学业成绩方面的影响。拟议的评估将侧重于2015/16年度在全州范围内随机推出的项目，并寻求回答三个核心问题：(a)在偏远部落地区和该州其他地区的公共教育系统中，学校层面的过程、投入和学习成果有何不同？MPSQA干预措施在多大程度上改善了项目学校的校级流程和学习成果，这在偏远部落地区的学校和该州其他地区的学校之间是否有所不同；(c)实现和抑制项目效果的途径是什么（例如教师努力、更好的学校管理、更专注的教学实践以及学生出勤率和努力方面的改进）？在回答这些问题时，我们将寻求使用广泛的行政数据，包括全州范围内的低风险评估结果，以及独立管理的测试和详细的过程指标，这些数据是通过定期对随机选择的小学和中学的学校子集进行不事先通知的学校访问收集的。这些独立管理的测试将防止项目学校官方测试结果被操纵的可能性，并允许通过仔细的测试设计将测试结果与印度和国际上的其他评估联系起来。核心提案和研究设计是基于使用调查数据的定量影响评估方法。然而，认识到大规模的全系统改革往往具有意想不到的方面和不同主体和不同治理水平之间的多层相互作用，我们还提出了一个定性的组成部分，寻求对以下方面的补充细致理解：(a)干预运作的背景，就教育系统中相关主体(学生、教师、校长和教育公务员)(b)如何通过教育系统的这些不同层次来调节项目的效果，并了解这些改革的“促成因素”或瓶颈可能是什么。

项目成果

Improving Public Sector Management at Scale? Experimental Evidence on School Governance India

大规模改善公共部门管理？

• DOI: 10.3386/w28129
• 作者: Muralidharan K

Myths of Official Measurement:Auditing and Improving Administrative Data in Developing Countries

• DOI: 10.35489/bsg-rise-wp_2020/042
• 发表时间: 2020-07
• 作者: Abhijeet Singh

Improving Public Sector Management at Scale? Experimental Evidence on School Governance in India

大规模改善公共部门管理？

• DOI: 10.35489/bsg-rise-wp_2020/056
• 作者: Muralidharan K