Quantifying pandemic-driven educational assessment modifications: grade predictions, student and school characteristics, and university outcomes

点击翻译按钮获取中文摘要

In response to the Covid-19 pandemic, A-level and equivalent examinations were cancelled in 2020. Instead, students completing qualifications in 2020 were awarded grades based on teacher determinations, initially standardized by Ofqual via an algorithm, and eventually returning to so-called centre-assessed grades. Prior to the pandemic, teachers had predicted grades for students, including those applying to university. This is established practice, part of the normal procedure of applying to university. Evidence suggests though that teacher predictions can be inaccurate, often overestimating a student's true achievement, and can vary between subgroups (e.g. by gender, ethnic, socio-economic background; or by school characteristics, incl. rate of progression to university). The aim of this research is to quantify the change generated by the 2020 assessment procedure that replaced examination-based awarded grades with what amounts to another set of predictions by teachers, after initial attempts to apply a standardization approach. The research will evaluate how it may be associated with student and school characteristics; and assess implications on progression to university for students and schools.The research will use a new data resource, linking information about students, schools, assessment results, and university applications, for 2020 and three years before the pandemic. These linked data, held jointly by the Department for Education, qualifications regulator Ofqual, and university application service UCAS, offer a unique opportunity to understand how any assessment modification affected students and schools in different circumstances.Despite the importance of predicted grades for university admissions, the evidence on their determinants and impact is limited. This research will address this gap in three ways.First, using the new data resource, the research will explore for the first time the relationship between UCAS predicted grades and awarded grades at the individual level. This will measure the prediction accuracy and explore how it varies by individual, subject, and school characteristics.Second, using innovative analysis, the research will contrast the prediction accuracy in the 2020 and the pre-2020 cohorts, quantifying the modification introduced by the 2020 assessment method, by subject and school. As long as no substantial variation in the prediction accuracy is observed across the pre-2020 cohorts, any change in this relationship in the 2020 cohort, over and above regular variation between years, can be attributed to the 2020 assessment procedure. The research will generate evidence around factors associated with the modification, including school type, size, composition, location, and other features. The modification will be also quantified using the 2020 Ofqual-provided calculated grades. This will create a further counterfactual for what could have happened were grades not awarded by the 2020 procedure, also accounting for student and school features.Third, the research will explore how the above-quantified modification has affected the university application outcomes of students and schools in the 2020 cohort compared to pre-2020, linking this work to existing policy efforts to widen access to university. The research will offer an insight into the impact of the 2020 assessment procedure on the types of educational outcomes that are associated with later employment and social mobility opportunities.Overall, the research will provide a comprehensive investigation of the relationship between predicted and awarded grades and the impact of the Covid-19 driven changes on this relationship, broadening the understanding of the determinants of inequalities in university access during the pandemic, helping inform policies to mitigate the negative impact of the pandemic on students' life chances, to inform policy that may improve outcomes contributing to social mobility.