Nine to one - not as easy as ABC

I blogged at the Sutton Trust on the dangers of an increasingly complex accountability system

Back in 1995, I helped David Blunkett commit a heretical act – at least in the eyes of the teaching unions. With the help of the late David Frost and a closely argued column in The Times, we embraced the need for school performance tables. Yes, we would look at improvement and not just absolute results, but we would still publish both to hold schools accountable and to inform parents.

Over two decades on and we have a lot more data available to us. Admittedly some of it – the detail in the invaluable National Pupil Database – is restricted to those meeting stringent data protection rules. But parents can access a pretty good summary of how well a school is doing on the DFE website. The only problem is that it has become a lot more complex. And confused.

That confusion can only have been increased by the latest announcements from the DFE this week. This year is the first time that pupils will be judged on a new 1-9 scale, replacing the current A*-G scale. The idea is that this will allow finer judgements at the top where gaining a 9 will be a lot harder than an A* - indeed, Tim Leunig, the DFE’s chief analyst, mused to his Twitter followers that only two pupils in the country might get all top marks in the new system.

But it is not at the top that the confusion and concern has been concentrated. Rather it is at the borderline. An important feature of the new system was supposed to be an ending of the focus by schools on the dreaded D-C borderline. I’ve always been slightly bemused by this concern: after all, a C is far more impressive to an employer than a D and it is deluding young people to pretend that their E is of any use to them at all. There does, of course, need to be more focus on encouraging Bs and As, but as a minimum the C grade was a reasonable one.

And despite the introduction of Progress 8 – the hugely complex statistical measure of progress on which schools are now supposed mainly to be judged – yesterday’s news shows that the C grade remains important. Teachers have been struggling for months to understand whether a score of 4 or 5 will see them over the line in the new system.

Ministers had previously indicated that key school targets would focus on the tougher 5 grade – a good pass – but pupils who gained a 4 could be eligible for progression to the sixth form or college. On Tuesday, Justine Greening tweaked this yet again saying that the performance tables will include two pass rates – those getting a 4 and above and those getting a 5 or above – particularly for the English Baccalaureate scores.

Confused? Parents will be. But more importantly, the whole thing threatens to undermine nearly three decades of school reform. Of course, the 5 A*-C measure was not perfect. But sometimes statisticians need to recognise that perfection may not be attainable if it reduces clarity. The data was a compromise, but with floor targets and minimum standards it did a lot to drive up standards, especially in the half of secondary schools where fewer than 30 per cent of pupils gained five good GCSEs twenty years ago. The danger is all this change makes it impossible to see where improvements are being sustained.

That matters to narrowing the attainment gap as well as to social mobility, because many of the schools which were performing badly in the past had a disproportionate number of disadvantaged pupils. Their decent results have spurred them to further improvement. Progress 8 is a tough sell to explain how well a school is doing because of its complexity and because of the distorting impact of a few individual pupils Now the nine to one scale is layered on top. Comparisons over time become meaningless and past successes may appear lost. All this at the same time as many of these schools bear the brunt of cuts and changes in funding.

I’ll be honest: I was a bit sceptical about the English Baccalaureate when it was introduced, in part because of concerns that it would hurt those improved schools. But research we published last year showed that it benefited early adopter schools and improved opportunities for poorer pupils. However, the target of 90% or 100% of pupils achieving it is not realistic, and the case for a technical option remains strong. But as a way of simply demonstrating a pupil’s or a school’s success in core subjects, it has proved to be not a bad idea. And crucially it is comprehensible.

But that is not the case with these latest changes. If even the head of the exams regulator admits that parents and employers will be “confused” by the new system, and that communicating what it means will be a struggle, there are real problems ahead. And it is not just individuals and pupils that could be the losers, it is the credibility of an accountability system that has delivered real improvements in our schools.