Exam Analysis – Leading a Department 1 (Annual)

An introduction

I have been teaching for ten years. In the grand scheme of things, not very long at all. I work with and talk to those who are into their third decade and whose experience far outweighs mine. It is these interactions, seeing great practice in others, being challenged by strong leaders that I feel has allowed me to pick up so much of what has worked within our educational setting, and obviously within my subject area – secondary science.

When I began working at my current school they were newly opened and starting their second year of operations. When it comes to the day to day running of our science department, I see its success underpinned by a very simple set of systems, thoughts and actions that I hope to convey over the coming weeks as a set of 20 or so blog posts.

I am hoping to convey the key areas that have worked for my colleagues, the pupils we guide as well as myself on this journey through the secondary science curriculum. By the end of these instalments (if you can make it that far), there will be easy to use examples and templates that I will provide through this website.

I have tried to structure these in such a way that you can pick them up and implement them into your day to day organisation, meeting agendas, leadership link meetings, lessons, telephone calls and so on. I am under no illusion that everything I am about to mention will work for you and your pupils, parents and/or colleagues, but hopefully there will be something that resonates with you or you want to take the risk with trying out for yourself in your own setting.


I have designed these posts so that you can easily navigate through the strategies, ideas and thoughts based on how often they must be addressed.

For example the second group of posts looks at what items you may want to consider once per year. This doesn’t mean they have to be done at the same time though. For example exam results analysis that we look into in a bit more depth will happen in August or September following their publication, whereas curriculum overview may happen closer to the end of the previous year in June or July. Both have a frequency of once per annum, but both happen at different times within the same school year.

The hope therefore is that any new, or veteran, department lead can pick up this book, find a particular thought or idea and see how to implement it into their own faculty area.


Part 1 – exam analysis

One of the biggest changes we made within our department was the exam analysis that we did. In my early years as a Department Lead there was little that went into this aspect of the faculty. With it being a new school we didn’t have any external assessments or GCSEs, just our in house, end of topic or end of year tests.

When exams did come about (our first set of GCSEs was the lsat year on the old-GCSE, letter graded specification). We received the grades on that Thursday . We felt bad for pupils who didn’t quite hit the mark, we felt ecstatic for those who had ‘over achieved’ (this was always more their feeling than ours – you always have to believe otherwise how can you get out of bed every morning and teach?) and everything else fell in the middle. We looked broadly at the spread of grades and often one or more of the following comments would be passed between the science staff;

– “How did they get that grade”
– “Did you see how well A. N. Other did on their chemistry paper?”
– “Great pass rate with this year’s cohort”
– “Biology didn’t get as many grade nines as we had predicted”
– “Did you see the progress score of Miss Smith’s class?!”

Then once we had come down from our high, or picked ourselves up from our lows it was back to business. The curriculum was much the same as were the resources, assessments, department specific CPD, technology used, websites linked to, YouTube videos used and so on. Also having been a relatively new school, we never had a set of external exam results with which we could delve into with much detail due to the fact in year one we only had pupils in year seven. However that all changed after our first set of results and the new level of analysis we could do in house, with our data manager, in department, using simple spreadsheets and externally, with the bank of tools and data made available by exam boards.

the question i hope to answer

What analysis can you do with your summer exam results and How this analysis can help you plan the following year’s materials.

THe answer to that question

The exam board that I am most used to due to the courses I have taught is AQA. It is important to note that the features of AQA may not be shared with other exam boards and vice versa. Also, there may be fees or charges included with AQA (or other boards) functionality or analysis. Whoever you use to assess your students I would highly recommend speaking to their subject support team to see what they can do for you between results day in August and the start of the new school year.

Grade Overviews

What can you do with this? See how many of your students gained particular grades and compare with the previous year, results from similar centres or the national average for AQA centres.
How does this help you the following year? Most staff will instantly want to know, what was their pass (9-4), strong pass (9-5) or other key performance indicator (KPI). The grade overview can break this down for you. How did you do compared to national, then you might have to do a bit more digging. Each year, JCQ collates the collective results for its members from more than 26 million scripts and items of coursework.

Marks Analysis

What can you do with this? See your average performance by subject group, specification, individual paper and individual questions. With skills and topics analysis you can see marks for different assessment areas, for some components.
How does this help you the following year? This is particularly useful when you want to see how pupils have done on different papers within the same exam series. For example in science we can see the difference between marks achieved for biology paper 1 and biology paper 2. There is also a break down by Assessment Outcome/Objective (AO). Again this is useful, for example for us after our first set of results we noticed pupils scored very high in the recall questions but poorly in the higher thinking skills, suggest, explain and so on. We could then change our pedagogy accordingly so that our assessment points, questioning and feedback focussed more on these areas to try and close that gap.

Student Results

What can you do with this? View the results for a particular candidate, see how close they were to grade boundaries and compare with other results, similar centres and the national average of AQA centres. You can see the question paper, mark scheme and examiners’ report for each exam.
How does this help you the following year? This is sometimes the most time sensitive of all the items mentioned in this table. For some pupils, they may be only a few uniform mark scale (UMS) points away from achieving the next grade. This download allows you to see what score/UMS pupils achieved across their papers and as a result how far their overall subject grade is from the next above or below. This is an important piece of information as you can then suggest to your SLT and/or exams officer not only which pupils should be entered for a remark but also which paper you would want to resubmit. While costly (you are refunded if there is a change to the grade) I honestly believe that schools should ring fence some of their budgets each year solely for this expense.

Download Results

What can you do with this? You can download any or all of your results as an Excel spreadsheet or CSV file to analyse offline or use in your school systems.
How does this help you the following year? The most basic analysis starts with you being able to access the data. Here you can download the results and run whatever manipulations that you see fit. It could be a case of identifying outcomes and measures for particular pupils, groups or otherwise.

Maintain Groups

What can you do with this? Set up your own groups of students and you can see comparisons for teaching groups, birth month, free school meals or whatever you choose.
How does this help you the following year? The ability to do this allows you to dig a little deeper into the data presented. You can set up teaching groups and then run comparisons between the classes. Are there any teachers who could do with some CPD support or potentially others who were able to help secure better outcomes for pupils and from whom you could learn or share best practice? This group maintenance section is key.

Examiner Reports

What can you do with this? Reports on exams written for examiners. They have not been written with students in mind and are written, to an extent, in exam-speak. The exam boards are trying to make the reports more user-friendly. They offer a unique insight into the exam series just undertaken and common errors seen across the whole country.)
How does this help you the following year? Examiner reports hold a lot of very useful information within them. As an exam marker myself I know how much the mark scheme can be amended following an initial period of marking and feedback to lead assessors. The examiners report is where the board is able to feedback common gaps in skills, knowledge and application back to subject teachers nationally. We have often been able to read through the examiner report and on our curriculum identify the lessons where we need to go into more detail or include certain aspects because the feedback from the board has identified it as a weakness nationally. Even if it is a strength for your staff and the examined cohort, any insight into the mind of an exam marker or indeed exam writer is very useful.

Access To Papers

What can you do with this? It is possible with the consent of the pupils and payment of a fee to get access to a specific pupils suite of exam papers from the most recent examination series.)
How does this help you the following year? This is a great idea that I have seen done at primary school. The primary would keep the three pupils’ books who they feel would set a high standard for incoming pupils the following school year. They were also the examples they would show prospective parents as to what the standard was like at the school. We use them on the one hand to show pupils what a grade 9 set of responses might look like as well as professionalism of presentation and so on. We also can use the answers of our pupils to help with staff training. So that less experienced staff, or those who are not exam markers for the board, can get the hands on actual exam condition answers. They also prove very useful when giving pupils examples to critique or to use in class. There is a different atmosphere when pupils know they are looking at a response from a pupil in the school as opposed to one that was written by the ‘all-knowing’ teacher.

Resources

Below is a department toolkit to help ask the correct questions linked to summer exam analysis as well as the next steps. Again the toolkit is not exhaustive, merely an aide-memoire for the process.