Wednesday, 2 May 2012

What lies behind the claims of dumbed down exams?

The exams regulator, Ofqual, has inherited a programme of exam standards reviews that were introduced in 1997 with the support of both the outgoing Conservative and incoming Labour governments. Yesterday, they produced a veritable collection of these reports that were duly spun to a grateful media as confirmation of a concerted dumbing down drive over the last decade.

I decided to look more closely at what Ofqual actually said. And what it does say poses rather more interesting questions than the critics would care to ask.

  • There has been a move away from essays towards multiple choice questions on some papers. In other words, a move away from analysis towards testing knowledge of facts. Now, in many ways, I happen to agree that this is a retrograde step. But isn't the tenor of what the Government has been saying that we need more facts and fewer skills. Won't the result be that we move towards more multiple choice questions like this and fewer 'more demanding' essay style questions? 
  • A big complaint of the dumbing down brigade on science was directed at an effort to make the subject more relevant and interesting at GCSE through 'How Science Works'. Fascinatingly, Ofqual tells us that in GCSE Biology "this change did not affect the demand of the qualification overall." and in Chemistry it even made it more demanding. (Earlier reports on physics and on general science had been very critical in 2009)
  • In GCSE maths, the big change that made a difference was moving from a three-tier to a two-tier qualification, set at intermediate and higher levels. But the problem with tiered papers is that teachers can under-estimate a student's ability and enter them at a tier where they cannot excel. Tiering can be an enemy of aspiration, in other words. The problem when the tiers are removed is that the exam boards reduce demand. There must be a better way to do this that measures true attainment without capping aspiration. That has always been the policy goal.
  • In several courses, OCR seems to have been more demanding than AQA, once again showing the perverse effects of having competing examination boards for the same subjects.
  • Reviewers judged A2 Geography to be less demanding because of the removal of the coursework element. Coursework - typically a 4,000-word investigation - was an effective way to assess skills by, for example undertaking and reporting on investigative fieldwork
So, let's be clear of the implications:
  • First, coursework has been removed or reduced because of concerns over cheating, and the extent to which it can be manipulated by parents or helped by teachers. It is often set now under supervised classroom conditions. But it is not a synoptic assessment in a high stakes exam setting. So, a way must be found to allow for such investigations in ways that avoid cheating.
  • Second, competition between exam boards, far from promoting 'innovation' seems to provide schools with easier options. Ofqual has set its face against moving towards a national exam board for A levels and GCSEs. But shouldn't it think again?
  • Third, we clearly need fewer multiple choice questions and more essay-style questions. That does mean more markers and more expensive exams. It also means more time devoted to learning the skills of essay writing and research. All to the good. But it needs a Secretary of State who is clear that he sees this as being as important as acquiring facts.
  • Fourth, there is not necessarily a tension between making science more relevant and maintaining standards (provided it isn't accompanied by lots more multiple choice questions).
  • Fifth, the system of reviewing standards over time seems to work. And it has existed now for 15 years.

It's always helpful to have a little context.


Tegsie said...

I never understood why there were separate exam boards. If comparison of results is to be made, students should be taking the same exams, set by a single provider with a single policy on marking.

Dave said...

The idea behind more than one examination board is that it introduces competition. The change was introduced by the previous conservative governments who believed that competition and market mechanisms generally increase choice and improve standards.

Charlieman said...

If you wished to enter Birmingham or Manchester University, it was convenient to have studied JMB A Levels. The universities wrote the exam papers to make it easy to judge applicants for their courses. It also meant the under grads had a common entry level -- common ability.

Times have changed and perhaps a common set of exams might be pertinent.

John said...

If it was suggested that the driving test was opened up to competing market forces there would be an outcry. Why does it not invoke the same response for the exams that shape our young peoples lives?
Market forces in the context of national exams is a nonsense. The only difficulty for Labour to argue this is that they didn't grasp the nettle when they had the opportunity, it would however be a good way of reducing costs with no negative frontline impact.
Such a policy would have overwhelming support of all concerned, with the possible exception of Pearson's.

matt said...

Dave is incorrect; exam boards developed independently and were originally regional or associated with a particular university or other organisation. They have gradually merged so now there are three big ones (AQA, OCR, and Edexcel - a part of the Pearson publishing group), WJEC which is the Welsh board but can be used by English institutions, and a vast array of bodies providing vocational qualifications. In picking boards, any sensible head of department looks at its student cohort, the nature of the course and exam, and the stats, and picks the board that will provide the most students with the best outcome. This depends on lots of things and is less simple than "the easiest" but for example, a lot of factual questions can be a problem for A grade students because grade boundaries at the top end will be very close together so a few bits of missed revision can cost a student a Russell Group place. This seems like a good argument for amalgamation. On the other hand, there's no doubt that Edexcel have massively improved their customer service over the last few years - would they have worked so hard on this if everyone had to use them regardless?