How much do schools matter?

The government’s strategy for improving educational outcomes focuses on improving schools’ performance.  But how important are schools in explaining variance in pupil attainment? There is surprisingly little evidence on the issue.  We sought to address this question in an article for the February 2018 edition of the National Institute Economic Review, examining the attainment of half a million pupils in more than 3,000 English secondary schools over the period 2009/10 to 2015/16.

To our knowledge, this type of exercise has not been undertaken on this scale since analysis published by the Department for Education and Skills for 2003. Much has changed in the school system since then. Some changes – notably the academies programme, which takes schools out of local authority control, replacing it with substantial autonomy for school leaders over hiring, firing and paying teachers – might have led to greater variance in performance across schools. Others, such as the inspection regime devoted to driving up standards and helping schools identified as ‘failing’, may have reduced variance in school performance at the bottom of the distribution.

Our study shows that schools do matter for pupil performance, although, in line with previous studies, including the Department’s own study, they account for only a minority of the variance in attainment.

During our study period, the headline indicators used in the school league tables, and the way they are calculated, changed regularly. This matters because the amount of variance in pupil attainment accounted for by schools differs according to the measure of attainment used.

The current headline pupil attainment indicator, Progress 8, which is used in the current school league tables, compares attainment at the end of primary schooling with attainment at the end of compulsory schooling, typically at age 16, so tracks the progress of pupils. We find that in 2015/16 schools explain 14 per cent of the pupil-level variation in Progress 8. Part of this ‘school effect’ is related to the similarity of pupils within schools, so that pupils with characteristics associated with better or worse attainment cluster in schools. Our analysis shows that controlling for the similarity of pupils in schools, schools account for around 9 per cent of variance in pupils’ attainment.

Up to 2014/15 school league tables were based on the percentage of pupils in each school who achieved five or more A*-C grades in GCSE or equivalent qualifications, including English and maths. In this period, for the five or more A*-C grades indicator, schools explain a similar share (10-12 per cent) of the pupil-level variation as for the Progress 8 indicator in 2015/16. However, if we focus on attainment indicators that are consistent over time, based only on GCSEs, we find schools have a much larger influence on the variation in pupil scores, as much as 35 per cent (30 per cent if previous attainment is taken into account).

These findings are broadly consistent with earlier research. The DfES research found that 20 per cent of the variation in end of secondary schooling attainment was explained by the school in 2003; when a “progress” indicator was considered this fell to 8-13 per cent.

We do not identify any great increase in the importance of schools in explaining pupil attainment in England since 2003, despite an increased focus on school level interventions. Furthermore, the majority of pupil attainment is not accounted for by schools. Other studies point to other important factors such as genetics (Shakeshaft et al., 2013) and family and neighbourhood (Nicoletti and Rabe, 2013) as important predictors of pupil attainment. Whether recent government reforms such as the roll-out of pay flexibility for school teachers, can increase the proportion of pupil attainment accounted for by schools remains to be seen.

You can read more about this research in our February 2018 Economic Review (subscription required)

David Wilkinson is Principal Research associate in University College London’s Department of Social Science and Research Fellow at the National Institute of Economic and Social Research.

Alex Bryson is Professor of Quantitative Social Science in University College London’s Department of Social Science.

Lucy Stokes is Principal Economist at the National Institute of Economic and Social Research.