ATAR Notes: Forum

VCE Stuff => Victorian Technical Score Discussion => Topic started by: Joseph41 on April 30, 2018, 03:13:53 pm

Title: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Joseph41 on April 30, 2018, 03:13:53 pm
(https://i.imgur.com/ymzToTK.png)

Recently, Australia’s chief scientist, Dr Alan Finkel, suggested the ATAR is “completely obscure” and “is leading to students being given poor advice” in regard to their subject selection. [1]

After six years of high school and about that many working for VCE organisations, I can’t help but agree that the ATAR and technical side of VCE isn’t that transparent. It’s generally not taught in schools, it’s confusing for students, and I’m quite sure it leads to a lot of unnecessary anxiety through high school.

The point of this post is to try to allay some of those concerns, and to spell out clearly how VCE works from a technical point of view.

Note:
None of this is crucial knowledge. Like, you don’t need to know this stuff to do well.

What is an ATAR?
Spoiler
Let’s start from the start. The ATAR (Australian Tertiary Admission Rank), renamed in 2010 from the ENTER (Equivalent National Tertiary Entrance Rank), is a number calculated by VTAC to allow tertiary institutions to compare students’ levels of performance across subjects. [2] Universities and other tertiary institutions can use the ATAR as one criterion for entry to various courses. ATARs are reported between “less than 30” and 99.95, which is the maximum ATAR. There is a minimum increment of 0.05 (so, ATARs of 30.00, 30.05, 30.10, 30.15… 99.80, 99.85, 99.90, 99.95). No ATAR ending with anything other than a second decimal place of 0 or 5 exists.

The ATAR does not reflect your raw marks. That is, an ATAR of 80.00 does not indicate you have averaged 80% across the year, nor does it indicate you achieved 80% on your exams. Instead, it suggests VTAC deemed you to have outperformed 80% of your cohort (those you started Year 7 with) – it’s a percentile ranking. VTAC explains the ATAR as “the estimate of the percentage of the population in the relevant age group that the student outperformed”. [3] That is, where you ranked in your cohort – not your raw marks on tests and exams. It’s for this reason that you can’t achieve an ATAR of 100; you can’t outperform 100% of your cohort (which includes yourself).

The ATAR is not the only way to gain entrance into tertiary education, and it’s certainly not make-or-break. This largely falls outside the scope of this thread, but you can read more about the importance of otherwise of the ATAR here.

How is the ATAR calculated?
Spoiler
As you can probably imagine, it’s quite difficult to establish how well a single student has performed in context of the entire state. Why? Because that student will be exposed to different subjects, different SACs (School Assessed Coursework – more on this later), and different stringency of marking throughout the year. Different SACs and different approaches to marking both threaten a fair system. Naturally, VCAA wants a fair playing field, and doesn’t want to unjustly advantage or disadvantage any student. For this reason, we can’t just say, “we’ll add up all of your scores throughout the year, and rank you that way!”, because that wouldn’t necessarily be a true indication of students’ achievement. As such, we need a more objective way of ranking students across subjects and conditions, and for this we use the aggregate.

The “aggregate”
Spoiler
Calculation
Spoiler
Very many students are aware of the ATAR; many fewer are aware of the aggregate. According to VTAC, “[t]he aggregate is a number between 0 and a value that can reach over 210.” [4] As above, your aggregate is what VTAC uses to rank you against other students in the state. You can picture it as your “ultimate high score”, with each study score being analogous to your high score for an individual level of a game.

So, how is the aggregate calculated, exactly? Remember, we need this to be able to work out each student’s ATAR. Well, your aggregate is basically the sum of:
•   your highest-scoring English subject (English, English Language, Literature or English as an Additional Language);
•   your next three highest-scoring subjects; and,
•   if applicable, 10% of your fifth and sixth highest-scoring subjects.

That is, a maximum of six subjects can count toward your ATAR. Your top English subject plus your next three best subjects form your “top four”; your fifth and sixth subjects are referred to as your “bottom two”. The top four (or “primary four” in VTAC language) is required in order to receive an ATAR; the bottom two is not. Anything beyond the sixth subject (so, say you studied a seventh, eighth, ninth etc. subject) is not considered toward calculation of the ATAR. [5]

NOTE: It is not only Unit 3/4 VCE subjects that can count toward your ATAR. You also have the option of:
•   “Higher Education Studies” (colloquially, uni extension subjects), which can count as either your fifth or sixth subject. An increment of 3.0 – 5.0 is awarded depending on achievement in that subject; more information can be found here.
•   Alternatively, VCE VET sequences (either scored or unscored). The processes for unscored VET subjects are a little different, but they essentially add to your ATAR based on your other scaled study scores. [6]

Example
Spoiler
So, let’s demonstrate how this top four/bottom two jazz works. Let’s take an example of Joseph, who studies seven Unit 3/4 subjects. Note that these subjects don’t need to be completed in Joseph’s year of graduation in order to contribute to his ATAR. Joseph’s subjects, plus his final scaled scores (more on this later), are listed below.
•   Biology: 46
•   Chemistry: 25
•   Health & Human Development: 49
•   Literature: 28
•   Physical Education: 33
•   Specialist Maths: 42
•   Studio Art: 30

If we order these from best to worst, we’re left with:
•   Health & Human Development: 49
•   Biology: 46
•   Specialist Maths: 42
•   Physical Education: 33
•   Studio Art: 30
•   Literature: 28
•   Chemistry: 25

But remember that an English subject has to be included in the top four, so let’s put Literature (Joseph’s only English subject) right to the top by default
•   Literature: 28
•   Health & Human Development: 49
•   Biology: 46
•   Specialist Maths: 42
•   Physical Education: 33
•   Studio Art: 30
•   Chemistry: 25

From these scores, we can now calculate Joseph’s aggregate. Remember, that’s the top four subjects (including an English), plus 10% of the fifth and sixth subjects if applicable. So, in this case, that would be:

28 + 49 + 46 + 42 + 3.3 (10% of Physical Education) + 3.0 + 10% of Studio Art) = 171.3.

Note that the score of 25 for Chemistry doesn’t count at all, because it was Joseph’s seventh subject.

(https://i.imgur.com/xHBwnCr.png)

Conversion to ATAR
Spoiler
Now, this aggregate of 171.3 probably doesn’t mean much to you – nor should it, because aggregates are rarely spoken of or considered. But what now happens is all aggregates are ordered, giving VTAC a way in which to rank students, irrespective of what subjects they took. Each student, based on their aggregate, is then attributed a percentile rank, and this percentile rank is converted into an ATAR. [7]

How do they do this? Well, they have what’s called an “aggregate to ATAR conversion table”. It is how it sounds – it just shows what minimum aggregate is needed in order to achieve whichever ATAR. You can find the 2017 aggregate to ATAR table here. But to give a general idea of what’s required:

(https://i.imgur.com/Y9VQiv3.png)

Restrictions on the ATAR
Spoiler
Just a quick note here: there are a couple of other things that might affect you, too. For example, there is a limit to the number of similar subjects that can count toward your ATAR. VTAC says:

Quote from: VTAC
There are some restrictions on how certain combination of studies may be counted for an ATAR.

In each of the study areas of English, mathematics, history, contemporary Australian studies, information technology, languages and music:

•   at most two results can contribute to the Primary Four
•   at most three results can contribute to the ATAR, the third being counted as a 10% increment for a fifth or sixth subject

This also includes Higher Education studies.
[8]

So, let’s say you’re studying the following subjects, and get these scaled results:
•   Further Maths: 50
•   Maths Methods: 44
•   Specialist Maths: 42
•   Biology: 41
•   English: 36
•   Uni Extension Maths: 3.5

In this case, there are four maths subjects (Further, Methods, Spesh and uni extension). Even though the first three are your best three subjects, only two of them can count in your top four. And because only three maths subjects can count toward your ATAR, the uni extension subject won’t count at all. So basically:

•   English: 36 [top 4]
•   Further Maths: 50 [top 4]
•   Maths Methods: 44 [top 4]
•   Biology: 41 [top 4]
•   Specialist Maths: 4.2 [10% increment]
•   Uni Extension Maths: N/A

Okay, great. So now we know that your ATAR is determined by your percentile rank, and that your percentile rank is determined by your aggregate. We also know that your aggregate is the sum of your study scores (with stipulations), which leads us to the question: what is a study score, exactly?

What is a study score?
Spoiler
A study score is sort of like a mini-ATAR: it’s a standardised score from 0-50 you’ll receive for each of your individual subjects. The median study score is 30. [9]

Like with the ATAR, though, study scores aren’t really scores at all – at least, not in the traditional sense. Again, they’re rankings of how you’ve performed in context of the cohort for that subject. So, a study score of 40 certainly doesn’t mean you’ve averaged 40% (or even 80%, as study scores are only out of 50).

Study scores are standardised, meaning they work on a bell-curve. The standard deviation is about 7, meaning that approximately two-thirds of students in a subject will receive between a 23 and a 37 study score. Anything below a 23 or above a 37 is more than one standard deviation away from the median study score of ~30. I’ve modified this image below to show that standard distribution, with some study scores in red.

(https://i.imgur.com/svKJhzQ.png)

So you can see that the majority of students will receive a study score of between 23 and 37 (for any subject).

How are study scores calculated?
Spoiler
The basic process is this:
•   You are awarded a letter grade (from E to A+) for each Graded Assessment. There are three Graded Assessments (or GAs) per subject. These will change from subject-to-subject. For example, for Further Maths (a subject with two exams), GA 1 refers to overall SACs, GA 2 is the first exam, and GA 3 is the second exam; for something like Psychology (just the one exam), GA 1 refers to Unit 3 SACs, GA 2 is Unit 4 SACs, and GA 3 is the exam.
•   These scores all contribute to a total score of sorts, which is standardised (against the bell curve mentioned earlier) by VCAA, taking into consideration things such as weighting (as not all GAs are worth the same amount).
•   The score is converted to be out of 50. [10]

The standardisation means that, every year, the distribution of study scores is pretty close to this: [11]

(https://i.imgur.com/XHHKwPB.png)

So, as you can see, only ~9% of students receive a study score of 40+, and only ~2% of students receive a study score of 45+. To receive a raw study score of 30, you need to be in the top ~half of the cohort.

What this means is that, in theory, you probably could roughly predict your study scores if you had all of the necessary information (as you could roughly work out how many standard deviations you were from the median). Here is an old ATAR Notes thread on the matter, but like, quite honestly, I think it’s a waste of time. I know it can be frustrating not really knowing where you sit etc., but ultimately, intricately going through the system and trying to predict your scores won’t actually increase them. Studying might, so that’s a much better use of time!

Now, you might be thinking, “okay, cool – but like, not all SACs are the same; how is that fair?” Well, it wouldn’t be if we didn’t have SAC moderation.

SAC moderation
Spoiler
VCAA wants a fair system. The current system reflects that; it has been devised in a way that aims to eliminate all unfair advantage or disadvantage given to any particular student. As you can no doubt imagine, there are a couple of things that threaten the fair system that VCAA is hoping for.

One of these is the fact that different schools have different SACs and could, therefore, give out extremely easy or extremely difficult tests compared with the VCAA standard. For example, my school might give really easy SACs, and the school in the next suburb might give very difficult SACs. This would obviously be to my advantage – it would be easier for me to get a good mark than those students at the other school.

Another potential flaw in the system is the idea that teachers mark differently. So even if, hypothetically, every school in the state administered the same SACs, we’d still face the issue that some teachers mark stringently, whilst others are more lenient. It would genuinely suck to lose points from your results just because of the way your teacher marked, and VCAA knows this very well.

So, what VCAA does is aligns (or moderates) each school’s SAC results to be more in line with VCAA’s own standards. Let’s consider a few different examples.

(https://i.imgur.com/yCDqCac.png)

We have three different schools here: School A, School B and School C.

Throughout the year, students from School A received a median mark of 50% on their SACs. But in the end-of-year exam, the median mark for that cohort was much higher: 75%. What this suggests is either that School A’s SACs were more difficult than VCAA’s standard, or that they were marked too harshly. This wouldn’t be fair on the students from School A, so VCAA boosts their collective SAC marks accordingly.

Throughout the year, students from School B received a median mark of 80% on their SACs. But in the end-of-year exam, the median mark for that cohort was much lower: 50%. What this suggests is either that School B’s SACs were easier than VCAA’s standard, or that they were marked too leniently. This wouldn’t be fair on other students who sat more difficult SACs, so VCAA lowers School B’s collective SAC marks accordingly.

School C’s median SAC mark and exam mark is about the same, so VCAA makes no change; those SACs already seem to be reflective of VCAA’s standard. This directly from VCAA: [12]

Quote from: VCAA
Students now have moderated School-based Assessments, which are comparable across the entire VCE system. For example, if we compare moderated School-based Assessments in Geography, we can be sure that a student in Horsham with a mark of 75 has a higher achievement than a student in Richmond with a mark of 60. Before moderation we would not have been able to tell which student had the higher achievement because they are on different scales…

Comparing SAC marks before moderation just isn’t logical. In the same way, consider two tubs of bricks: one weighs 50 kilograms, and the other weighs 100 pounds. It’s illogical to say that the second tub of bricks is heavier just because 100 is greater than 50; we can’t really compare them, because they’re set on different scales (kilograms and pounds). When we convert them both to the same measurement scale, we can actually see that 50kg is heavier than 100lbs, but we couldn’t tell that until we’d moderated the scale.

That’s exactly what VCAA is doing with SAC moderation. As such, comparing your raw SAC marks with those of people you know from other schools is a pretty fruitless exercise. It’s a little more complicated than that, as indicated by the quote below, but that’s the basic premise.

Quote from: VCAA
[T]he following marks are determined for both the School-based Assessment scale and the external score scale: the highest achievement, the upper quartile, the median and the lower quartile.ł These scores are used as fixed points for aligning the two scales…
[13]

The other take-home point here is that, really, it’s in your interest for your cohort to perform well on the exam(s), and trying to sabotage your cohort’s exam marks is really just sabotaging yourself.

Where it gets really interesting, though, is that this sort of thing also happens on an individual level, and that’s what we’ll look at now in the form of rankings. Also note that these raw study scores don’t consider the competitiveness of that subject in that given year, and we’ll consider this in the “Subject scaling” section later on.

SAC rankings
Spoiler
You might have heard on these forums that “it’s not really your raw marks that matters; what matters is your ranking.” What does this mean?

Well, throughout the year, a hierarchy is established within your cohort for SAC performance. That is, the student in your cohort with the best SAC average will be rank #1, the next best average will be rank #2 and so on. A lot of things can change through SAC moderation/scaling etc., but SAC ranking is not one of them. So, if you finish as rank #1, you’ll stay as rank #1.

Quote from: VCAA
[T]he rank order of students’ assessments in a school is not altered by statistical moderation… The VCAA acknowledges that teachers are in the best position to measure students’ academic achievement in School-based Assessment.
[14]

Having a high rank is important because it means you benefit from your cohort’s exam performance. This is most easily explained with an example.

(https://i.imgur.com/UDlRkF9.png)

(Please note that the following section is very rough, and in reality it doesn't work as neatly as this. Having the second highest ranking doesn't mean you'll get the second highest exam mark as your scaled SAC mark etc. - it's just an easy way to explain the basic process. Actually considered are gaps between ranks and other factors, so keep this in mind.)

Okay, so there’s a bit to take in here. Basically, this table shows a cohort of 10 students studying one subject – it doesn’t matter which subject. They’re listed in by SAC ranking – so, Student A had the highest average, Student B had the next highest average, then Student C, and so on. It also shows each student’s end-of-year score on the VCAA exam.

What you might notice straight away is that the best-performing students from throughout the year in this cohort didn’t actually perform the best on the exam, as may be the case in reality. But to run you through what happens:

•   Rank #1 gets the best exam score as their moderated SAC score. In this case, Student A achieved a 52% on the exam, but benefitted from Student D’s 99% as they’re rank #1. Student A’s exam score isn’t affected, but they get 99% as their moderated SAC score.
•   Rank #2 gets the second best exam score as their moderated SAC score. In this case, Student B achieved a 96% on the exam, which just happened to be the second best score of their cohort. As such, their moderated SAC score is 96%.
•   Rank #3 gets the third best exam score as their moderated SAC score. In this case, Student C achieved an 89% on the exam, which just happened to be the third best score of their cohort. As such, their moderated SAC score is 89%.
•   Rank #4 gets the fourth best exam score as their moderated SAC score. In this case, Student D achieved a 99% on the exam. However, as this was the best exam score of their cohort, it actually goes to rank #1. Student D gets allocated 85% as their moderated SAC score, as this was the fourth best exam score.

As you can see, having a high ranking can be a pretty big advantage. If you’re stuck down near the bottom of your cohort, your moderated SAC score is, at least to a degree, a little contingent on exam performance of those ranked above you.

Quote from: VCAA
Statistical moderation does not change the rank order of students, as determined by the school’s coursework scores. A student given the top score for coursework by his/her school will have the top score after statistical moderation, no matter how they perform on the examination(s).
[15]

Now, once all this has been said and done to find your raw study scores, they get scaled according to competitiveness.

Subject scaling
Spoiler
As we know, VCAA is looking for a fair system, and a big consideration in that is how to compare vastly different subjects. For example, how can you really compare Visual Communication & Design, and Chemistry? Very difficult.

The idea is that the raw scores you receive don’t actually take into account how difficult it was to achieve a mark in the middle of the cohort in various subjects. For clarity – and this is something VCTAC also stresses – by “difficulty”, I don’t mean how inherently difficult the subject is, or how hard the content is. After all, those are extremely subjective measures. Instead, “difficulty” here is measured by competitiveness in the subject.

Quote from: VTAC
All Study Scores are scaled to adjust for the fact that it is more difficult to obtain a high Study Score in some studies than others. This is NOT because some studies are inherently harder than others. This IS because some studies attract a more competitive cohort of students than others.
[16]

This is the reason that subject scaling changes a little from year to year: not because VCAA is arbitrarily changing its mind on how difficult subjects are, but based on varied competitiveness in subjects from cohort to cohort. There is nothing inherently about Specialist Maths that means it scales up, and it could just as easily scale down if the cohort were right. Similarly, Health & Human Development could scale up if the cohort were right.

Now, how is “competitiveness” measured? Basically, by how well students in a particular subject do in their other subjects. This might seem a bit counter-intuitive, but let me explain it in this way.

Say you’re going to compete in two different 100-metre races. In the first, you’re up against a bunch of slow, old, overweight non-athletes. In the second, you’re up against Usain Bolt, Tyson Gay, Yohan Blake, Asafa Powell, and a bunch of other professional sprinters. Neither of these races is inherently more difficult – they’re both simply a 100-metre running race. But the competition is vastly different, thereby making it much easier to achieve a mid-place ranking in the first race than the second.

The competition, here, is determined by how well the competition does in their other races. So, Bolt, Gay, Blake, Powell etc. are, on average, going to do very, very well in their races; the non-athletes from the first race, probably not so much. The same sort of principle holds with the VCE system.

So, according to the 2017 scaling report, English Language scaled from a raw 30 to a scaled 33. To get to this point:
•   We take all of the English Language students studying the subject that year.
•   We look at how those students have performed in all of their other subjects.
•   We take whatever the EngLang students averaged in all of their other subjects, and that becomes the new average for English Language.
•   So, in this case, it was determined that the English Language cohort performed above the average by about three points, which is why the subject scaled up by three.

Equally, we could look at, say, Geography, which scaled from a raw 30 to a scaled 29.
•   What we can extrapolate from this is Geography students, on average, scored a 29 for their other subjects.
•   And the average study score for Geography, therefore, has been scaled down by one to account for this below-average competition.

Some people see subject scaling as rewarding or punishing those taking certain subjects, but it’s simply not the case. All VCAA is doing is levelling the playing field. Without this scaling, those studying subjects where it’s more competitive and, therefore, more difficult to achieve a middle ranking, would be unfairly treated.

With this in mind, the best thing you can possibly do is to choose subjects you a) need as pre-requisites for your university course; b) are passionate about and enjoy; and c) are good at. There is no way to “game” the VCE system in this way.

Proviso: all of this is just based on my understanding. There could be errors. Please point them out if there are.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Lear on April 30, 2018, 04:23:45 pm
My understanding was that even if you do terrible in school SACs, a great exam score can boost your terrible sac scores. I’m not quite sure I understand your information completely but there appears to be no mention of such changes. Is there more of a myth then ?
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: darkz on April 30, 2018, 04:30:58 pm
My understanding was that even if you do terrible in school SACs, a great exam score can boost your terrible sac scores. I’m not quite sure I understand your information completely but there appears to be no mention of such changes. Is there more of a myth then ?

Well from my understanding, if everyone in the cohort does well in the exam, then your SAC ranking won't matter as much since you just get the nth highest exam score for your final SAC score (So if you have a really strong cohort and you end up rank 30, but the 30th highest exam score was still like 90%, then you get that 90% for your SAC score which isn't too bad)
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Sine on April 30, 2018, 04:33:33 pm
My understanding was that even if you do terrible in school SACs, a great exam score can boost your terrible sac scores. I’m not quite sure I understand your information completely but there appears to be no mention of such changes. Is there more of a myth then ?
Yeah that thought process isn't necessarily 100% true I think that line is just said so people don't lose too much hope tbh.
Your great exam score on it's own won't increase your sacs but if you do well on the exam and a lot of people in your cohort do well it could drag your sacs up a little.

Well from my understanding, if everyone in the cohort does well in the exam, then your SAC ranking won't matter as much since you just get the nth highest exam score for your final SAC score (So if you have a really strong cohort and you end up rank 30, but the 30th highest exam score was still like 90%, then you get that 90% for your SAC score which isn't too bad)
rank 30 would not be a bad sac ranking in a cohort where the 30th highest exam is 90% so I wouldn't say those are terrible sacs
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: vox nihili on April 30, 2018, 04:36:35 pm
Dr Finkel also suggested that the scaling of subjects leads people away STEM subjects. Does anyone know what he was trying to get at there? My understanding was that it did the complete opposite.


@J41 I'm aware you may have already seen me rant about this on Twitter, soz :p

EDIT: also this post is great
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Joseph41 on April 30, 2018, 04:41:51 pm
Dr Finkel also suggested that the scaling of subjects leads people away STEM subjects. Does anyone know what he was trying to get at there? My understanding was that it did the complete opposite.


@J41 I'm aware you may have already seen me rant about this on Twitter, soz :p

EDIT: also this post is great

I thought the same!

In fact, I started a thread about it here. Would love your thoughts.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: PhoenixxFire on April 30, 2018, 04:58:25 pm
One thing I've never understood is that if your ATAR is calculated in comparison to your year 7 cohort how does that work if you finish VCE early or late? Surely it would have to be calculated against your year 12 cohort?

Also are SAC scores scaled the year we do the subject or the year we graduate? Because we only get scaled scores when we graduate but if we do the subject earlier doesn't that mean that they aren't being scaled properly? Or are they scaled against the correct year, just later?

This is awesome!!
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Sine on April 30, 2018, 05:03:03 pm
One thing I've never understood is that if your ATAR is calculated in comparison to your year 7 cohort how does that work if you finish VCE early or late? Surely it would have to be calculated against your year 12 cohort?

Also are SAC scores scaled the year we do the subject or the year we graduate? Because we only get scaled scores when we graduate but if we do the subject earlier doesn't that mean that they aren't being scaled properly? Or are they scaled against the correct year, just later?

This is awesome!!
not sure about your first one - you'd think they just use the cohort in which you "graduate with" there shouldn't be any significant differences across such a large sample size.

SAC scores are sacled the year you do the subject - since you sat the same exam i that year (the only standard measure across the state)and also SAC scores scaling does not occur in conjunction of subject scaling they are two entirely different processes and again subject scaling is done in the year you do the subject ( ask any year 12 with the same study score to know what you got scaled to).
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: PhoenixxFire on April 30, 2018, 05:07:13 pm
SAC scores are sacled the year you do the subject - since you sat the same exam i that year (the only standard measure across the state)and also SAC scores scaling does not occur in conjunction of subject scaling they are two entirely different processes and again subject scaling is done in the year you do the subject ( ask any year 12 with the same study score to know what you got scaled to).
Ah I wrote the wrong thing. I know sac scores get scaled the year we do it, I meant study scores. Thanks!
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Lear on April 30, 2018, 05:16:18 pm
How significant is the GAT? I have heard some mixed opinions about it.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Poet on April 30, 2018, 05:24:41 pm
How significant is the GAT? I have heard some mixed opinions about it.
The GAT (General Achievement Test) was made as an external measure to ensure that you are being marked accurately by your teachers and assessors in school. So if you are being marked too harshly or too lightly, the external assessors of the GAT may pick up on this through your GAT results and adjust your scores accordingly. Note that this is only for quite noticeable cases. The GAT doesn't typically count towards a person's ATAR/VCE results, but you still want to study and prepare to do the best you possibly can. So honestly it's not a huge stress, but definitely something you want to do your best in.
Here's VCAA's page on the GAT. :)
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: userrrname on April 30, 2018, 06:27:52 pm
I have a question. By moderated SAC scores, do you mean student A gets the highest exam mark as their moderated SAC score, meaning their exam score stays as 52%? Or do they get the 99% for their exam score as well? Because the first option doesnt necessarily mean the highest study score, right?

Or will stuffing up SACS significantly affect your study score? So working hard towards your exam seems fruitless if you're ranked at the bottom and others stuff up lol
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Sine on April 30, 2018, 09:11:02 pm
I have a question. By moderated SAC scores, do you mean student A gets the highest exam mark as their moderated SAC score, meaning their exam score stays as 52%? Or do they get the 99% for their exam score as well? Because the first option doesnt necessarily mean the highest study score, right?

Or will stuffing up SACS significantly affect your study score? So working hard towards your exam seems fruitless if you're ranked at the bottom and others stuff up lol
only the sac score changes the exam score always remains the score that you yourself achieved. Rank 1 doesn't necessarily get the highest study score and in large cohorts often they do not get the highest study score.

Yes stuffing up sacs will impact your study score but that will all be dependent on cohort strength/performance/state performance. Idk about "fruitless" - doing well on the exam will increase your SS but it would be higher if you aced your sacs as well.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: userrrname on April 30, 2018, 09:29:15 pm
only the sac score changes the exam score always remains the score that you yourself achieved. Rank 1 doesn't necessarily get the highest study score and in large cohorts often they do not get the highest study score.

Yes stuffing up sacs will impact your study score but that will all be dependent on cohort strength/performance/state performance. Idk about "fruitless" - doing well on the exam will increase your SS but it would be higher if you aced your sacs as well.
Ohh okay, at the end of the day though exam marks (at least in some subjects) outweigh sac marks, so essentially if you're ranked 1 it will benefit the sac aspect of your result but stuffing up the exam will still affect you significantly. It gets so confusing when people start saying things like rank 1 will get the highest mark no matter who gets it, or if rank 1 can't get 50 then no one can.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: userrrname on May 01, 2018, 08:33:12 pm
So If i do really good on the hard stuff basically the exam and i prepare my back end off for that and i m 3rd, but the top ranked person does alot worse than me just because the knew how vce works, then he would get my mark simply because of the 'rank'. He can walk in not worrying about anything and just doing average on the exam and i study extremely hard for a good exam score because I have been told that Exams weigh harder and i want a good exam score for myself? how does the system differ that situation?
Yeah but apparently he would ONLY get your amazing exam mark as his moderated sac scores, meaning his 'average' exam mark will pretty much remain. So, amazing sac marks + average exam mark = his study score. Not his rank 1 sac marks + your amazing exam mark.

Well at least this is what I've understood.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: alanyin1 on May 02, 2018, 10:41:26 pm
I'm not sure how useful this will be, but here were my marks and such when I graduated from MHS in 2016 in Chemistry. I ordered the Statement of Marks, so these should all be 100% accurate. Keep in mind that MHS' cohorts are fairly strong, so most people in my year level would probably safely assume that their SAC marks be moderated upwards. Also, please keep in mind that many (if not all) of my comments, regardless of how I try to support them with my results and regardless of how objective I try to sound, are just suggestions and speculation. Anyways, the following numbers were provided in the attached photo.

A couple of things to note regarding the results:
-After talking with teachers during the year and after results came out, there was speculation that Unit 4 SACs were worth slightly more than Unit 3 SACs. Of course, this is only speculation.
- A+ Range for exam was 198 to 240 out of max. 240
- I assume that the term 'Moderated SAC Scores' is the same thing as 'FINAL' in the photo. I will similarly interpret SCHOOL as the same thing as 'raw marks'.

My SAC ranking was definitely not 1st place, as I did not win my school's internal subject prize, given to the student with highest SAC marks (and performance on our end of year practice exam used to differentiate between those who have the same total SAC marks), however my SAC ranking was definitely somewhere in the first 10 ranks out of ~130+.

Going off the previous discussion, does the 'moderated sac scores = exam scores arranged from highest to lowest, with nth exam score correlating to nth SAC ranking' statement still hold true? If it is true, how does it work for subjects which only have one exam (i.e. GA1 = U3 SACs, GA2 = U4 SACs)? Because as you can see, neither the SCHOOL or FINAL are the same for any of the numbers, which implies that they are separate results, not the same result 'copy-pasted' to GA1 and GA2.

Additionally, I personally felt that an exam score of 227/240, albeit safely within the A+ range, is pretty low to score a 50 in any subject. Although I have no way of comparing with my peers for their exact figures, as I was one of the few, if not the only person, who actually bought the Statement of Marks (which is $10+ as far as I can remember), I suspect that not many people, if at all, would have performed better on the exam in my cohort - I dropped just an average of 6.5 marks out of 120, turning into a 227/240 average marked by 2 examiners.

Anyways, converting my exam score into a percentage (94.583%), how then is it possible for me to receive a 94% and 98% SCHOOL/raw SAC mark ('unmoderated') for my SACs if I was not the first rank, let alone higher for the FINAL (presumably the 'moderated') results? A 98% would mean a 235.2/240 on the exam, which although possible, is extremely difficult - probably worthy of Premier's tbh. Moreover, even if this were true, how could I receive a FINAL/moderated SAC score of 100/100 for U4, if the presumption is that the 'highest exam score goes to the first rank's moderated SAC result'? What does the word 'moderated' mean in the context of this discussion and in VCAA's processes, because wouldn't our assumed definition imply that at least someone in my cohort got 240/240 (100%) on their exam, and, moreover, since I was not first place, that more than one individual received this result, so that I could receive moderated SAC scores of 100/100? Finally, if even this is true, I find it extremely hard to believe that someone who can get 240/240 on the exam does not have decent enough SAC marks to warrant a study score of 50, or at least a higher proportion of 49s, 48s, 47s etc, because the next highest SS was a 48, maybe another, then 47, then decreasing. Probably, what this suggests, if the previous discussion is ALL true, is that a relative exam result percentage is provided, and not the physical number (so you getting 95% on the exam does not mean that someone else will get the 'same 95%' as their moderated SAC score). What I am suggesting is that the whole cohort's performance in the exam compared to the SACs will have a greater influence on the SCHOOL --> FINAL change rather than individuals outright taking others' results. For example, if your cohort, on average, does 'as well as expected' as suggested by SAC results, then your cohort will get good scaling. If, however, your whole cohort gets like average A+ on SACs but then on average B on the exam, then everyone, on average, will be dragged down. How this affects certain individuals is not clear from this hypothesis.

Funnily enough, despite only this subject getting a 50 for me, three of my other subjects (Methods, English and Spesh all had FINAL SAC scores of 100/100, which only further complicates the problem - and yes, I purchased Statement of Marks for all my subjects lol).

Since VCAA will never release their processes, we can only assume what really goes on. In my opinion, I don't think that moderation has as much of an effect as the prior discussion suggested, even though all the information that was told to me when I was in Year 12 has remained exactly the same (at least for my results). Certainly, there is no way for me to justify this opinion, due to the effect of MHS' strong cohort and my relatively high rank inevitably moderating my results upwards. However, my opinion is that the degree of moderation to SACs really only affects the SCHOOL --> FINAL mark difference. For example, NOT being a higher rank MAY HAVE meant that my Unit 3 SACs only scaled from 94/100 to 99/100, not 100/100 (or is this +5 score change fixed for everyone in the cohort, with 100 being the cap? How can we verify the validity of this?). Additionally, the cohort's performance on the exam relative to their SAC marks IS what probably determined the 94->99 change. My views are obviously subjective to my own experiences, so take what I say with a grain of salt.

But my personal opinion is that as long as your SAC ranking is decent (the location 'decent' depends on what Study Score you are aiming towards), and you perform really well on the exam (low - mid A+), then you will be more or less guaranteed the result that you are aiming towards, regardless of your school's performance on the exam as a whole (which, under the prior discussion, would drag your SAC marks downwards). E.g. if you get rank ~5 out of like 100 and a 90% on the exam, you should be expected to get around 45, and at least a low 40, if you get rekt by adverse scaling depending on your cohort's performance.

Finally, the most important aspect of this whole process, and, really, the whole of Year 12, is to just enjoy it all and to do as best you can. Obviously, all this scaling stuff is a concern, but it shouldn't be a concern until like just after your exam. Besides, it's only just the start of May right now, so there is still loads of time. If you do poorly on a SAC, then improve in time for the next one! Ask your teachers for feedback and identify weaknesses and strengths, not just in the subject, but in your studying methods. Practice makes perfect, as cliched as it is. If you focus on doing well and have a growth mindset, then you will naturally reap what you sow. The ideal goal, really, is to be happy with your subject score/ATAR and not ponder about what-ifs (can't really say this with a straight face as I was desperate for +0.11 aggregate but we'll overlook this for now haha).
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Joseph41 on December 04, 2018, 02:44:18 pm
Bumping for benefit of those in the Class of 2019 and beyond. :)
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: aspiringantelope on December 31, 2018, 10:59:11 am
Soo what correlation does the Exam score have to do with the Moderated SAC Score?
Why is the Exam Score used to determine moderated sac score according to ranking?
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Lear on December 31, 2018, 11:12:45 am
Because internal sac scores vary in difficulty, strictness and content. If we just used internal scores students who received easier assessments would be greatly advantaged. The external exam and the GAT are the ONLY assessments that are standardised across the state and hence are used for moderating internal sacs. It just makes it fair.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: aspiringantelope on December 31, 2018, 11:45:32 am
Because internal sac scores vary in difficulty, strictness and content. If we just used internal scores students who received easier assessments would be greatly advantaged. The external exam and the GAT are the ONLY assessments that are standardised across the state and hence are used for moderating internal sacs. It just makes it fair.
Oh I see, so schools have different SAC difficulty ? or are they all the same?
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Lear on December 31, 2018, 11:55:08 am
Different.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: aspiringantelope on December 31, 2018, 12:04:53 pm
Different.
Ugh I thought VCAA distributes same SAC LOL
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Sine on December 31, 2018, 12:09:57 pm
Ugh I thought VCAA distributes same SAC LOL
VCAA don't play a part in what questions your teachers will choose for SACs. They will have some general advice on what to do with weak/strong cohorts but the overall difficulty/questions is up to your school to decide.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: aspiringantelope on December 31, 2018, 01:35:01 pm
VCAA don't play a part in what questions your teachers will choose for SACs. They will have some general advice on what to do with weak/strong cohorts but the overall difficulty/questions is up to your school to decide.
I see.
Thanks!!
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Highway_end on December 16, 2019, 07:36:38 am
Hi, I was just wondering if the normal distribution of study scores was for raw or scaled scores?
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Bri MT on December 16, 2019, 08:39:52 am
Hi, I was just wondering if the normal distribution of study scores was for raw or scaled scores?

Raw
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: naun81 on December 16, 2021, 04:29:50 pm
Hi, after reading this topic I understand a bit better how it all works, thanks!
But can anyone advise on the school choice?
I will be in year 9 next year, so school choice is kind of important. The problem is that there is no very good public school in my area and parents are unlikely to pay top private school fee. So my choice would be between average public and average private school.
If I understood correctly, in a weak cohort it will be easy to get high rank and keep your good exam mark to yourself, which is good.
However, if average cohort results are bad this will negatively affect my score.
Does it all even out in the end?
Do you know anyone who got high ATAR (>99) while attending average public school?
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Joseph41 on December 16, 2021, 04:34:26 pm
If I understood correctly, in a weak cohort it will be easy to get high rank and keep your good exam mark to yourself, which is good.
However, if average cohort results are bad this will negatively affect my score.

I think this is a pretty fair conclusion speaking in general terms, yep, but every situation is different. Others might have more to say or different opinions.

Quote
Does it all even out in the end?
Do you know anyone who got high ATAR (>99) while attending average public school?

I went to a public school that was pretty much bang on average overall (a touch above average in terms of median study score) and managed a 99.65.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Sine on December 16, 2021, 04:37:47 pm
Hi, after reading this topic I understand a bit better how it all works, thanks!
But can anyone advise on the school choice?
I will be in year 9 next year, so school choice is kind of important. The problem is that there is no very good public school in my area and parents are unlikely to pay top private school fee. So my choice would be between average public and average private school.
If I understood correctly, in a weak cohort it will be easy to get high rank and keep your good exam mark to yourself, which is good.
However, if average cohort results are bad this will negatively affect my score.
Does it all even out in the end?
Do you know anyone who got high ATAR (>99) while attending average public school?
If you are in a weak cohort it is important to rank 1 (or rank very highly) if you don't it can decrease the types of scores that you get. If you are rank 1 in a bad cohort your cohort results don't matter that much as long as you do very well on the exam.

There are definitely people who get high ATARs from average public schools (see above). Whilst selective/some private schools generally get more high ATARs this is more to do with those schools having more students being of high SES rather than the actual school being better/worse.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: Joseph41 on December 16, 2021, 04:40:21 pm
If you are in a weak cohort it is important to rank 1 (or rank very highly) if you don't it can decrease the types of scores that you get. If you are rank 1 in a bad cohort your cohort results don't matter that much as long as you do very well on the exam.

There are definitely people who get high ATARs from average public schools (see above). Whilst selective/some private schools generally get more high ATARs this is more to do with those schools having more students being of high SES rather than the actual school being better/worse.

This is a better response than mine. Good post. 👍
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: blueycan on December 16, 2021, 07:08:27 pm
Hi, after reading this topic I understand a bit better how it all works, thanks!
But can anyone advise on the school choice?
I will be in year 9 next year, so school choice is kind of important. The problem is that there is no very good public school in my area and parents are unlikely to pay top private school fee. So my choice would be between average public and average private school.
If I understood correctly, in a weak cohort it will be easy to get high rank and keep your good exam mark to yourself, which is good.
However, if average cohort results are bad this will negatively affect my score.
Does it all even out in the end?
Do you know anyone who got high ATAR (>99) while attending average public school?

Sine and Joseph41 answered super well but also thought I'd mention that I attended a low SES school ranked pretty poorly (around 450, maybe even lower) and I ended up getting a 93.75– I know friends with 97 and 98 as well in my cohort. It comes down to your individual rank, results, and ultimately your drive to do well over what school you go to.
Title: Re: VCE Behind the Scenes: Moderation, Rankings, Scaling, Aggregates
Post by: naun81 on December 16, 2021, 07:29:31 pm
Thanks everyone:) I think biggest challenge when attending low rank school is that you are not sure what is a high standard. You may be a top of class, praised by teachers and relax, while in a high rank school you would be in a middle and tried to push yourself further. Probably need to go to tuition to have some comparison  :D