Thursday 26 January 2012

Can you have too much data?

Nobody is a greater believer in the power of data than I am. During the Labour government, the real power of the tests and tables introduced by the Tories was unleashed, with success in raising minimum standards in schools and maximum NHS waiting times. The coalition have recognised the value, at least, of the former. But we also made the mistake of trying to have too many targets. Specifically, the Treasury started to impose its endless whims on departments, leading to some real problems, as with crude exclusion targets; some that seemed doomed to fail, like the bid to cut truancy; and many that were an irritant for departments and an unwelcome addition to their box-ticking for frontline professionals, but had little impact on service users.

With today's league tables, there is every chance the coalition are embarking on the erroneous course. I'm all in favour of the data.gov.uk initiative, but having tried to open some of the extraordinary zip files buried within it, I do wonder quite how accessible it all is. On a smaller scale, today's tables present the same problem: we now know how well schools progress; how well they do at the top end, at the bottom end, and for average pupils; we now how well they do in Gove's favourite subjects, as well as in the subjects already reported separately of English, Maths and Science; we know the progress a school has made; and its level of improvement. All of this is valid and potentially useful. But we are told that this barrage of statistics will see schools avoiding 'gaming' which is apparently all that an improvement from 35 to 58 per cent in the proportion of students gaining five good GCSEs (or equivalents) including English and Maths since 1997 amounted to. Yet the DFE's own statistics show that even excluding BTECs that are worth four GCSEs, the total is 53 per cent this year for GCSEs or 52 per cent for academic GCSEs. Hardly all the result of gaming. Indeed sponsored academies and the London Challenge - two Labour programmes regularly praised by Gove - played a pretty large role.

But the Government is becoming a bit confused here. In fact, as Michael Gove's adoption of them recognises, schools responded to floor targets, particularly thosr for the five good GCSEs, precisely because they were deemed the most important in lifting them from poor to average. I hear from heads that the schools most likely to adopt the EBacc are those that are satisfactory and see this as a route to good. Again, these two measures are acting as floor targets. The danger of trying to judge schools on a much wider range of criteria through league tables - as opposed to Ofsted judgements which should do so - is that the parent has more information than they can absorb, and they are unable to process it. Five good GCSEs may have been a flawed measure, just as the KS4 English and Maths tests may not be perfect, but it is a simple and meaningful way of judging a school. When CVA was introduced, some schools saw it as an alternative to good GCSEs, yet few employers will look kindly on a clutch of Ds and Es - it may have shown progress but it was often not enough for those students, who needed real qualifications. I only hope that the perverse effect of the government's information overload is not a slowing of the pace of improvement.

1 comment:

JoeN said...

I would recommend you read this piece of research, "Data dictatorship and data democracy: understanding professional attitudes to the use of pupil performance data in English secondary schools," by Anthony Kelly
Christopher Downey and Willeke Rietdijk from the University of Southampton. It might give you pause for reflection.