What happens if a top student/rank 1 performs badly on a SAC?

[ArtyDreams]

Just had a question purely out of curiosity (and also because I like knowing how things work ah ha)

So say we have a cohort (doesn't matter the subject) which ranges from abilities. There are a couple of top scoring students who are really strong, and will most likely perform really well on the exam.

Say theres Rank 1, a really strong person, they were doing really well, and just happened to perform reallllyyyy badly on a SAC as they were having a bad day, causing their rank to fall down a fair bit.

Is this a disadvantage to the entire cohort - that a strong person is ranked in the middle - or a benefit? Depending on the teacher, can this person be moved up because thatll be an accurate representation of the rankings?

Hope this question made sense. I'm just curious! 

[Sine]

Just had a question purely out of curiosity (and also because I like knowing how things work ah ha)

So say we have a cohort (doesn't matter the subject) which ranges from abilities. There are a couple of top scoring students who are really strong, and will most likely perform really well on the exam.

Say theres Rank 1, a really strong person, they were doing really well, and just happened to perform reallllyyyy badly on a SAC as they were having a bad day, causing their rank to fall down a fair bit.

Is this a disadvantage to the entire cohort - that a strong person is ranked in the middle - or a benefit? Depending on the teacher, can this person be moved up because thatll be an accurate representation of the rankings?

Hope this question made sense. I'm just curious! 

Ok so let's say there is a cohort of 20.

Rank 1 falls to Rank 10 after a bad sac and that is their final ranking.

For the exam we will assume everyone from the new ranks 1-9 and ranks 11-20 perform how they should on the exam. E.g. rank 1 has a higher exam score than 2, 2 higher than 3, etc etc. But Rank 10 (the strongest student) scores the highest.

This results in the rank 1's sac score now being determined by rank 10( the strongest students) exam score. Then there is a flow-on effect where rank 2's sac score is now being determined by rank 1's exam score (the second highest exam score). This continues until rank 9's sac score is determined by rank 8's exam score and then rank 10 (the strongest student's sac score) is determined by rank 9's exam score.

There is a slight shift.

So As you can see rank 1-9 benefit. As it is always beneficial to be ranked higher than your "true" rank in terms of exam performance. Rank 10 the strongest student takes a hit due to the low rank. Beyond that rank 11-20 all get there sac score determined by their own exam score. This is unlikely to happen in a real cohort though.

The biggest detriment to a cohort's performance is not a strong student ranking low but a strong student messing up the exam and underperforming.

As for manipulating sac marks, I don't think that is the best idea. I know teacher's do it but they really shouldn't. The sacs and exam are different things. If the point of the sacs were to just align student in terms of how they go on the exam you could just chuck out the sacs since they have no relevance then. Or as a teacher just rank your students based on a practice exam sat 1 week before the actual exam.

On the cohort level, ranking students based on how they will go on the exam is probably the most advantageous if you are just thinking about scores (although I don't think it is right to push someone's rank down if they do well on sacs but you don't think they will do well on the exam).
On the individual level having a ranking that exceeds your actual ability/knowledge is advantagous.

[ArtyDreams]

Thanks for this sine - really helpful!

Just one more question because I keep getting varying answers to this from my teachers.
Is it possible/advantageous to have multiple rank ones?

My methods teacher said you really cant (and I think so too, otherwise how does the SAC scaling work?) but some other teachers think its good to have multiple rank ones. For example, I have a teacher thinking that multiple rank ones will help the entire cohort.

[keltingmeith]

Thanks for this sine - really helpful!

Just one more question because I keep getting varying answers to this from my teachers.
Is it possible/advantageous to have multiple rank ones?

My methods teacher said you really cant (and I think so too, otherwise how does the SAC scaling work?) but some other teachers think its good to have multiple rank ones. For example, I have a teacher thinking that multiple rank ones will help the entire cohort.

So like - it's worth noting that SAC scaling in general is a black box that we don't know exactly how it works. Everyone thinks they know how, a few people have things they swear by, but most of what we know is essentially just gossip passed from student to student to teacher to student to "my uncle works at VCAA and he said" to... You get the point. It's not that we have no idea how it works - VCAA has published some information on how it works - but we don't know the exact picture, which is important when dealing with "what if" scenarios.

I'd imagine the teachers want to have mutliple rank 1s because moderation tries to preserve the scale between students. However, if there are multiple rank 1s, which one is the actual top? Well, probably whoever had the best exam - and so the best possible exam result is then used to bring everyone up. But I don't (and I don't think anyone really does - no matter how much they might claim to) know if this will lead to a more advantageous position. For example, some have purported that in the case someone does better than rank 1 in the exam, then the person who did better gets their mark, and rank 1 is then propelled higher by what VCAA would've expected them to get - in this case, you would want there to be only one rank 1, because then if rank 5 gets the top exam mark, ranks 1-4 will get even higher marks. In this situation, having multiple rank 1s would be disadvantageous.

Is it possible to have multiple rank 1s? Yeah, easy. If your teacher writes a stupid easy SAC where everyone gets 100%, well they have to report that everyone got 100%. Is it helpful in the long-term? No clue, and tbh the system is shrouded in enough mystery it's probably not worth losing much sleep over.

[Sine]

Thanks for this sine - really helpful!

Just one more question because I keep getting varying answers to this from my teachers.
Is it possible/advantageous to have multiple rank ones?

My methods teacher said you really cant (and I think so too, otherwise how does the SAC scaling work?) but some other teachers think its good to have multiple rank ones. For example, I have a teacher thinking that multiple rank ones will help the entire cohort.

If students have the same rank they end up with the same scaled sac score. However, I can't remember which scaled sac score they get. E.g. if there are two people on rank 1 they could either get the scaled sac score reflective of the highest exam score, the 2nd highest exam score or the average of the two. I think the average of the two would be the fairest since using the highest exam score would result in more teachers giving equal ranks.

However, whilst I can understand equal ranks within the middle/lower of the cohort a clear objective of a VCE teacher is to separate out the top students so that they all can get appropriate sac scores. E.g. in a strong cohort you would want to make your sacs harder so that not everyone is scoring 100%. On the other hand in a weak cohort, whilst not directly relevant to this topic, you would make your sacs slightly easier so students don't lose motivation.

Sometimes if students are on equal ranks teacher's will use tiebreakers such as a practice exam or bump up whoever they think will do better on the exam.

[chem-nerd]

Just one more question because I keep getting varying answers to this from my teachers.
Is it possible/advantageous to have multiple rank ones? 

Possible, yes
Advantageous, not so much

If students have the same rank they end up with the same scaled sac score.

This, and it's the average of the moderated SAC scores.  So advantageous for one student.

[keltingmeith]

Possible, yes
Advantageous, not so much

This, and it's the average of the moderated SAC scores.  So advantageous for one student.

Am curious if you have a source for this information?

[chem-nerd]

Am curious if you have a source for this information?

I did try to find my original detailed post for this from way back (edit: ~2009), but no luck.  This question has been repeatedly addressed at VCAA VCE Professional Learning for many years. 

It’s somewhat addressed in the FAQs https://www.vcaa.vic.edu.au/assessment/vce-assessment/how-vce-assess/how-pages/Pages/StatisticalModeration.aspx but they don’t publicly publish the specific examples.

edit: to add picture