Improving school governance and learning outcomes at scale: A randomized evaluation of the Madhya Pradesh School Quality Assessment program

点击翻译按钮获取中文摘要

We propose to conduct a randomized controlled trial (RCT) of one of the most ambitious recent attempts by a developing country government to improve education governance at scale. The evaluation will have a particular focus on the state of school governance, processes and learning outcomes in rural tribal areas, which have been one of the most challenging contexts to improve education in India.The Madhya Pradesh (MP) School Quality Assessment (MPSQA) intervention is a comprehensive program that aims to improve school governance and outcomes through a combination of (a) regular monitoring of schools, (b) creation of school report cards, and a customized school improvement plan, (c) quarterly follow-ups on progress against this plan, and (d) leveraging of ICT tools to collect and report real-time records of all assessment results, school improvement plans, and follow-ups to a dedicated online system. The PI's have worked with the project implementation team and have successfully randomized the rollout of the program across ~20,000 schools in the 2015/16 school year. The government is currently hoping to universalize the program to cover all 120,000 schools in MP by 2019. The timing of the evaluation is ideal: finding positive results may help accelerate the scaling up across MP and elsewhere in India, while not finding significant positive results (along with data on intermediate factors to understand reasons for non-impact) may prompt a re-evaluation of the program and its implementation before spending resources on a scale up that may not have the desired impacts on processes or outcomes.The evaluation will assess the effects of the program on key process metrics (e.g. teacher absence, effort and pedagogy) as well as student academic outcomes for two years. The proposed evaluation will focus on the randomized state-wide rollout of the program in 2015/16 and seek to answer three core questions: (a) How do school-level processes, inputs, and learning outcomes differ within the public education system between remote tribal areas and other parts of the state?; (b) To what extent does the MPSQA intervention improve school-level processes and learning outcomes in the program schools and does this differ between schools in remote tribal areas and other parts of the state; and (c) What are the pathways through which program effects are realized and inhibited (examples include improvements in teacher effort, in better school management, in better-focused pedagogical practices, and student attendance and effort)? In answering these questions, we will seek to use extensive administrative data including state-wide low-stakes assessment results, alongside independently-administered tests and detailed process metrics collected through regular unannounced school visits to a randomly-selected subset of schools at both elementary and secondary levels. These independently-administered tests will guard against the possibility of manipulation of official test results in program schools and also allow for the linking of test results, through careful test design, to other assessments in India and internationally. The core proposal and the study design are based on the use of quantitative impact evaluation methods using survey data. However, recognizing that large system-wide reforms often have unanticipated facets and multiple layers of interaction between different agents and different levels of governance, we also propose a qualitative component which seeks a complementary nuanced understanding of (a) the context in which the intervention operates, in terms of the interactions, perceptions and motivation of the relevant agents in the education system (students, teachers, headmasters and education civil servants) (b) how the program effects are mediated through these various layers of the education system and learning about what the 'enabling factors' or bottlenecks of such reforms might be.