In my first blog on this summer’s GCSE Maths results, I promised to write again taking a closer look at how the changes in this new GCSE have impacted on performance.
This is an overview of some of the things we tried to achieve with the first exams and a look at how well students did on various question types and at different levels of demand.
The three things I wanted to look at were:
- ramping of demand through papers
- performance on multiple choice questions
- performance across the three assessment objectives.
Ramping of demand
We have always designed GCSE papers with the intention that they should get more challenging as students work through them. However, we gave more attention to this in designing the new GCSE papers.
We did not want students to be put off by very difficult questions early in the paper and we did not want easier questions to appear late in the papers where some students may never reach them.
A simple measure of how effective this has been is to look at the average performance (as a percentage of marks available) for each four-page section of each paper. Ideally, this would decrease steadily across each paper and each paper in a tier should be similar.
Here is the Foundation tier:
It was good to see all papers starting and ending at similar difficulty. Perhaps the most interesting feature is the drop off in performance on papers 1 and 2 at the half way point in each paper.
As part of our internal review process, we will be looking at this more closely to see what we can learn for future papers.
Here is the Higher tier:
As with Foundation, the graphs show that ramping broadly worked and appeared to work best in paper 3.
It is interesting that, in two of the papers, students were slightly more successful at the last section of the paper than the penultimate one. We will also want to take a look at the particularly good performance in the second section of paper 1.
Performance on multiple choice questions
Leading up to the first exam, some teachers had expressed concerns about our use of multiple choice questions. They were worried that their students may not perform as well in these as they would in similar, short answer questions.
I looked at how the multiple choice questions were answered in comparison with all questions of similar demand. Examiners assign all questions to one of five demand levels, from low-demand at the start of a Foundation tier paper to high-demand at the end of a Higher tier paper.
I compared the average mark (as a percentage of the total available) for all questions at a particular level of demand with the multiple choice questions of similar demand. The results are shown in these two bar charts.
As you can see, the performance on multiple choice questions is generally very similar to performance overall, and there is no evidence here of students being disadvantaged by multiple choice questions.
The one anomaly is in the high demand questions where students appear to have done a lot better on the multiple choice questions. The reason for this is that there were only three multiple choice questions at high demand, across the three papers, and students answered two of them really well.
There are more multiple choice questions at low (tier F) and medium (tier H) demand so the evidence is stronger in those areas.
Performance across assessment objectives
Preparing students for the demands of reasoning (AO2) and problem solving (AO3) questions has been a significant concern for many teachers.
We have produced various resources, and led many workshops, to try and help teachers get to grips with the style of questions that may test each assessment objective. So, I was keen to see how students had performed across the three assessment objectives.
As with multiple choice, I looked at the average performance of each assessment objective at each level of demand. The results by tier are shown here.
The graphs are broadly as I expected. Students did better on AO1 than AO2 and AO3 at all demand levels, but not by much in most cases.
The most striking exception is around medium demand questions on the Higher tier where the dark green bar above is somewhat lower than might be expected, when compared with the medium demand questions at AO1 and AO3.
We will certainly take a look at that when we review question paper performance shortly, and it will be interesting to see how the situation changes over time as teachers and students become more competent and confident with the new assessment objectives.