Analysing HSC Results

HSC Analysis Can Be Worthwhile

Every school I know of analyses its HSC results in some way. That's a good thing, but only if the analysis is effective. I think an analysis has been effective if a teacher can answer three questions pretty well:

1. What are we teaching well?
2. Who are we serving well?
3. What are we going to do this coming year as a result?

Data analysis is work. There's no getting around it. But effective analysis that leads to positive change is meaningful work. The goal should always be to set reasonable expectations while making sure the results are meaningful and positive.

What follows is how I'd go about it, in the order the school year runs, with the tools on this site and the templates I use. I don't expect all of it to suit every school, subject or circumstance. The templates are Word and Excel files, so change them to suit you.

If you have templates of your own you'd like to share, I'd love to see them, or better still, share them in my Facebook group, NSW HSC Discussion for Teachers. I know there are rules about what you can share, but as far as it's allowed, education is at its best when educators share what works.

Term 3

The trials

I like the idea of taking a good look at student achievement in the trials. The trials are the last real chance to see where students are before the HSC, and the only one where there's still time to do something about it.

The HSC Trial Analysis Summary works like the HSC version. You write down what you expect, look at how students actually went, and then note the content and the students you want to follow up before the HSC exams. The most useful part is often the last: the students who didn't achieve what you'd expect of them, and what you'll do about it in the weeks left.

HSC Trial Analysis Summary Word

Term 4

Before the results

Once the exams are done, and before the results come out, write down what you expect. It's the step most people skip, and I think it's the one that makes the analysis in the new year richer and more worthwhile.

Get a paper copy of the exam and mark the questions you think your students were equipped to answer well, and the ones you think they found hardest. Keep it. In Term 1 the RAP tells you which question it is, not what the question was.

Then fill in Term 4: Before the Results, once for each class. It asks about the paper, the bands you think students are likely to achieve, students you expect to do better or worse than their assessment suggests, and groups of students you expect to achieve similarly. You'll download a Word document to keep for the new year.

It doesn't matter if you're wrong. I can't think of any results I've had where I wasn't surprised by some students. The point is to set expectations before you see the results, so that in Term 1 you're asking 'Is this what I expected?' rather than just 'How did they go?'

Keep the exam copy and your Term 4 documents somewhere you'll find them in January.

Term 4: Before the Results Web page

If you'd rather fill in a Word document by hand, the Expectations section of the HSC Subject Analysis Summary asks similar questions.

Term 1

What are we teaching well?

This question isn't just 'What are we teaching well?', but 'What are we teaching well that turns into exam marks in what's externally assessed in our subject?' The Item Analysis in the RAP is the best place to go for this. It's one of the best things NESA gives schools.

1. Start with your Term 4 predictions. The RAP tells you which question it is, not what the question was, so have your marked-up paper copy of the exam and your Term 4 documents beside you. Go through the Item Analysis and ask whether it played out the way you expected. Where it didn't, make a note.
2. Get the results out of the RAP. The Item Analysis has little overview built into it, so it helps to pull the results into a spreadsheet. The sample below shows how many students got each mark on each question, with a small bar in each cell. There are plenty of ways to do it, and teachers you know will have good ideas.
3. Compare with the state, but for consistency. The RAP also has an exportable table of your cohort's average on each question beside the NSW average. I don't really care how a cohort compares with the state on an individual question. What I care about is how consistently it compares across the paper. If students are usually a little above the NSW average but fall below in one section, that section is worth a look.
4. Look for bias and noise. Biased results lean one way: most students answering a topic well, or most answering it poorly. Biased results are a gift, because it's clear what to focus on. More often results are noisy, with no pattern: some students doing well here and poorly there, and others the other way around. Noise is harder to act on, and may point to underlying skills rather than particular content.
5. Embrace error. However big your school is, the cohort is too small to know anything for sure. You'll be more certain about some things than others, but there's no exact science here.
6. Analyse in conversation. Talk it through with your faculty. My thinking is at its best when I have to explain it to someone else. Be willing to be wrong, and change your mind when there's good reason to.

Item Analysis Sample Excel

Term 1

Who are we serving well?

By 'who are we serving well?' I mean how students achieved in our subject compared with their other subjects. HSC scores can't answer this. Don't try to answer it by comparing the share of Band 5 and 6 scores in your subject with other subjects. Bands are great, but they can't do that job.

The best way I know is to compare each student's achievement in your subject with their ATAR, on ATAR terms.

1. Get your students' ATARs. Students who apply through UAC now agree to share their ATAR with their school, so your school should have at least some of them. I've seen anywhere from about a third to all of them. The more the better.
2. Compare them on the Subjects page. Choose your subject, then paste each student's HSC score and ATAR into Your students: how were they served? It works out each student's ATAR Contribution and shows it beside their ATAR. Nothing you paste leaves your computer.

An ATAR Contribution answers the question, 'If I could study just one subject for my HSC and still receive an ATAR, what would my ATAR be?'

Ascribe value carefully. Just because a student achieved their best or worst result in a subject, there are no automatic value judgements to make. Student achievement is complex, and how any one student achieves has functionally infinite complexity. Be willing to see what you might improve about how you teach the HSC, and equally, don't bear burdens that aren't yours to bear.

If your school uses my reports or others like them, you'll probably have a measure like this already, for every subject at once. Mine also show how each subject has usually served students over the last five years. Reports for Schools

Term 1

What next?

There's no point analysing HSC results if there's no willingness for it to lead to change. The idea of continuous improvement is fraught. It's not that there aren't always gains to be had. It's that in a finite system, infinite improvement isn't possible.

So I think it's important for teachers to be able to say, 'These are the areas where, for now, we're fine with how things are,' and then name the areas where they're seeking improvement, and why.

Schools don't need to see all the work a teacher or faculty does. What they need are good answers to a few questions. The HSC Subject Analysis Summary is my version of that. Filling it in isn't the analysis; it's a summary of it. It starts with the expectations you wrote in Term 4, so the whole year's thinking ends up on one document.

It also asks about students' readiness for the course in the first place. When I've taught Maths Advanced, I've felt the deficit when students aren't ready for the heavy content we have to march through to finish the HSC in time. Every HSC subject has too much in it to teach to the depth teachers would like, so it helps to find the pain points and ask where, if anywhere, they can be eased.

For heads of faculty, the HSC Analysis Faculty Summary gathers each subject's summary into one place: the item analysis, how students were served and the top priorities for the next 12 months, then what's common across the faculty.

HSC Subject Analysis Summary Word HSC Analysis Faculty Summary Word

Context worth having on hand

These charts are there to give broad context to achievement. They're on the Subjects and Compare pages for every subject, and each one downloads as an image to print for your faculty.

• What Was Each HSC Score Worth? I reference this one all the time; I have it on a poster on my wall. Each dot is an HSC score, coloured by band, and how far across it sits shows what it was worth as an ATAR Contribution.
• What is a Band Worth? What achievement at a band level means.
• What Does it Take The inverted form of What Does it Mean, and better for conversations about a student's ATAR Aspiration. It shows the lowest HSC score that gave a fair contribution to an ATAR last year. The ATAR Aspiration page does this for all of a student's subjects at once.

Downloads

They're Word and Excel files so you can change them to suit your school.