ATAR Notes: Forum

VCE Stuff => Victorian Technical Score Discussion => Topic started by: keltingmeith on August 26, 2020, 03:58:16 pm

Title: Study Score Calculator (open beta)
Post by: keltingmeith on August 26, 2020, 03:58:16 pm
After opening the prediction thread to find multiple people asking for full sets of subjects, I snapped and decided to write an excel spreadsheet to calculate things for people because tbh it's a lot less effort lol

How to use the spreadsheet:
Right now, it's set to view permissions so nobody breaks it. At the top, click "File>Make a Copy" to make a copy of it to your own Google account. You can edit the spreadsheet there. Also, some of you might not be used to sheet terminology - when I say "cell", I'm referring to those rectangle-like boxes you can write things into.

From there, everything you need to do is in the first tab - titled "Study Score Calculator"

In the red cells, list your subjects. As you start to write your subject, it should appear on the list - select your subject from that list. You can also just click each arrow on the right of the cell and select your subject from the drop-down list. If your subject isn't there, I haven't added it yet. See below for why

In the green cells, list your subject scores AS PERCENTAGES. If you don't, it won't work. Note that the first column should have graded assessment 1, the second column should have graded assessment 2, and the third should have graded assessment 3. If you don't know what these are, look them up, they'll be listed in the study design of the subject - just open it up, and ctrl+f "graded" and it should pop up. Usually, GA1 is unit 3 SACs, GA2 is unit 4 SACs, and GA3 is the exam - but this is not always the case, particularly for subjects with multiple exams.

Finally, your predicted study score should appear in the blue cells.

I've included some examples on the calculator so you can see how things should be entered.

GA1 for mathematics subjects is slightly different and stupid, so see the second calculator in the yellow cells for how to calculate GA1 for a maths subject


Let it be known - it was taking far too long to do every single subject that VCAA offers, so I decided to just do a few I've seen being asked and left it at that. If there's a subject you want added, post below, and I will add it. But also, it's not hard to do, so feel free to add your own subjects yourself in the tab titled "Subject Grade Distributions 2019". You'll find the data you need in the 2019 grade distributions to fill everything out. The spreadsheet should be able to handle everything from there. (I'm likely still adding subjects as you're reading this topic lol)


How the spreadsheet works:
All of VCE basically operates on a bell curve. No, really, all of it. The reason for this is because of a statistical phenomena called "Central Limit Theorem". It's actually kinda really cool.

If we use the grade distribution data, we can then convert your graded assessments so that we know where you lie on the bell curve. This is called a "z-score"

By multiplying each z-score by their appropriate weightings and adding them together, this tells us where you'd lie on the subject's final z-score.

From there, we just multiply the final z-score by the mean and standard deviation of the raw study score (this is 30 and 7 for all subjects), and this gives you a predicted raw score.

Fun fact: this is almost the exact method VCAA uses to calculate your study scores. The only difference is they re-distribute your score after finding your final z-score, instead of just converting it to the raw score distribution like I do.


Limitations of this spreadsheet:
Nobody has pinpoint prediction accuracy. Even though this calculator will give you an exact score, take it with a grain of salt - expect it to be off by up to 3 points in any direction.

This method particularly has issues at the upper-end. This is because the graded distributions aren't true normal distributions, because they're cut off at 100%. To highlight these, you'll notice the Accounting example actually has 100% listed for every GA - and comes out with a final study score of 43. Ooft.

This method assumes your SAC scores won't change. The good news is, if you're rank 1, and your exam scores are higher than your SACs, your SAC scores likely won't scale down. However, if you're not rank 1, then it depends on how good your school is. If you're in the top 50%, and your school usually averages a raw score of more than 30 in that subject, your score likely won't go down. If you're not in the top 50%, or your school usually averages below 30 - that doesn't mean your SACs will go down, but I can't guarantee they won't go down. Sorry about that.

Like my fifth subject calculations that I recently posted - this spreadsheet uses data from 2019. This will change from year to year, so even if my calculator nails what your 2019 score would've been, if the distribution for your subject changes too much in 2020, then this score will be completely different to what your 2019 score was.


Finally, this spreadsheet is basically only for checking you're on track. Remember it's flawed, and if you're doing particularly well it's likely underestimating your score (for real - if you get 100% in all of your scores, and they don't change, you will get a 50 - not a 43).

(also, people who did VCE last year and have your scores saved somewhere, I'd love to use them to benchmark the spreadsheet to see how accurate/precise it actually is - send me a PM! And if the spreadsheet ends up being particularly bad, I'll make another post in this topic telling you all to stop using it)

Important to note: This spreadsheet is for 2020 use ONLY. The weightings are based on the ones in the 2020 study designs, which were changed because of COVID. Future years will require me to fix that part of the spreadsheet
Title: Re: Study Score Calculator (open beta)
Post by: Bri MT on August 26, 2020, 05:39:25 pm
If you are not sure what GA1, GA2, GA3 means it's usually as below or best approach is to do as asked and look it up for complete confidence; the study design is your friend here.

For subjects with 1 exam: GA1: u3 sacs, GA2: u4 sacs, GA3: exam
For subjects with 2 exams: GA1: sacs, GA2: exam 1, GA3: exam 2


Stickying this thread, thanks keltingmeith for the great resource!

Edit: has been pointed out that languages don't always obey the guidelines described above. Some do, some don't so check :)
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 12, 2020, 12:06:01 am
UPDATES:

Firstly, I've added some more subjects as they've been asked in the study score prediction thread, and one that was PMed to me. If you want more subjects, just tell me what you want added - it's much easier on me to add them by demand than it is to do all of them at once.

Secondly, there was a bug in HHD in that it was underpredicting your exam contribution. This has now been fixed - if you predicted a score for HHD before, you should predict again for your own peace of mind.

Thirdly, I didn't like that I was predicting a single score when I don't know how accurate the calculator is. So, I've changed the score calculator such that it now gives you a range of scores based on how far away you are from the mean. This should hopefully address the fact that my calculator is less accurate at the extremes, so the calculator should now be much more solid for higher study score calculations! (though it's still not perfect. High marks just inherently have higher variability, based purely on the way VCAA works their system) For those interested in EXACTLY what variability I'm currently using, I'm predicting 25% variability in the z-scores. This equates to anywhere from 1 point difference in the final score to up to 5 points difference depending on how far you are from the mean, and what subject you're doing.

Finally, I've now added a VET subject, by demand! VET subjects only have two GAs, so I've included a little note in the middle explaining how to input the numbers as a result. I've also included a note about low student numbers, as this marks the first subject added with numbers so low that it means the calculator is even less reliable.
Title: Re: Study Score Calculator (open beta)
Post by: homeworkisapotato on September 12, 2020, 11:20:59 am
Hey keltingmeith, thank you so much for doing this! I wanted to find my score for Bio, but I couldn't find it. I was wondering if you took it off or am I just super blind ahaha?
Title: Re: Study Score Calculator (open beta)
Post by: The Cat In The Hat on September 12, 2020, 11:30:28 am
Would it be too much to ask for Texts and Traditions (since it's a tiny cohort)?
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 12, 2020, 04:29:14 pm
Hey keltingmeith, thank you so much for doing this! I wanted to find my score for Bio, but I couldn't find it. I was wondering if you took it off or am I just super blind ahaha?

Bio is definitely still there! Maybe because the subjects were sorted in order I added them, not alphabetical order, you couldn't see it? I've fixed that now :) I've also added Biology to the list of examples for you, so you don't have to search if you copy a fresh sheet, hahah. You can also manually enter the subject name - you don't have to go by the drop-down list. The only problem is if you don't write the subject name EXACTLY as it is in the drop-down list, I can't guarantee you'll get the right results.

Would it be too much to ask for Texts and Traditions (since it's a tiny cohort)?

No cohort is too tiny for me to add - my metric for adding subjects is, "will at least one person look up a score with this subject?". T&T is now in the list - though be warned that because it IS a smaller subject, the calculator is going to be less accurate than it is with your other subjects. Not something I can help, I'm afraid.
Title: Re: Study Score Calculator (open beta)
Post by: The Cat In The Hat on September 12, 2020, 04:45:25 pm
Am I blind? I... don't see English? If it's not there could you please add it?

If you could add History: Revolutions that would also be nice :)

Thank you!
Title: Re: Study Score Calculator (open beta)
Post by: Geoo on September 12, 2020, 04:49:24 pm
Whilst you're at it, if you could add food studies as well, that would be great!

Thank you, I also really appreciate how you've made it for 2020 with the adjusted study design as well :)
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 12, 2020, 05:15:38 pm
Am I blind? I... don't see English? If it's not there could you please add it?

If you could add History: Revolutions that would also be nice :)

Thank you!

English was the first thing I added when I decided not to take the "add every subject" approach lol. List is also sorted alphabetically now, so you should be able to find it without much ease? Revs has now been added, too

Whilst you're at it, if you could add food studies as well, that would be great!

Thank you, I also really appreciate how you've made it for 2020 with the adjusted study design as well :)

Thank me after I then have to change it back for the old study designs lmfao, though I guess not a concern for you ;) Food studies is now added



EDIT: I have good news for everyone! Someone was nice enough to send me their results for methods, and the calculator was able to predict their study score within the precision I've set! This doesn't mean the calculator is working perfectly, but it's now proven to work for at least a higher-mid methods score. If anyone else is able to send me their score/results for any subject, please do - more testing will only help me in defining those precision values I've designed (and who knows - maybe this spreadsheet is more precise for some subjects, and I can lower their precision bounds to get more precise scores for people?)

Based on the result I have, I'm predicting within 20% deviation of the z-score. I'm keeping the bounds at 25%, because one test isn't good enough for me to want to change the precision parameter, but still a happy result!
Title: Re: Study Score Calculator (open beta)
Post by: homeworkisapotato on September 12, 2020, 05:36:37 pm
Bio is definitely still there! Maybe because the subjects were sorted in order I added them, not alphabetical order, you couldn't see it? I've fixed that now :) I've also added Biology to the list of examples for you, so you don't have to search if you copy a fresh sheet, hahah. You can also manually enter the subject name - you don't have to go by the drop-down list. The only problem is if you don't write the subject name EXACTLY as it is in the drop-down list, I can't guarantee you'll get the right results.
I'm so bad with technology, thank you so much! You're such a legend!!
Title: Re: Study Score Calculator (open beta)
Post by: The Cat In The Hat on September 12, 2020, 05:58:06 pm
Found a bug - playing around with HHD just to see what scores I needed to get the score I got, and found that if I put in 50% for all three the lower was 29 and the upper... 28...? Trying a couple of other subjects including English I found those values gave me a higher lower than upper bound (English it was 29 and 27). Thanks for doing this, but you might want to look into that?
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 12, 2020, 06:02:24 pm
Found a bug - playing around with HHD just to see what scores I needed to get the score I got, and found that if I put in 50% for all three the lower was 29 and the upper... 28...? Trying a couple of other subjects including English I found those values gave me a higher lower than upper bound (English it was 29 and 27). Thanks for doing this, but you might want to look into that?

Oops - yeah, I know exactly why it's doing that. Should be fixed momentarily, but I'll need to run some quick tests first. Check back in 15 minutes

EDIT: Make that 4 minutes, lol. All fixed.

EDIT 2: I now have the exact scores (as in, what actual percentages they got for each GA) for someone and the study score they got, and my calculator got the result bang on! So we have at least 1 very good calculation from it, lending it quite a bit more weight.
Title: Re: Study Score Calculator (open beta)
Post by: onlytherealonesknow on September 15, 2020, 05:27:45 am
well guess no one can get above 46 in further then lol unless they get bonus marks for writing a new question and answering it on the exam 😂😂😂
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 15, 2020, 09:48:42 am
well guess no one can get above 46 in further then lol unless they get bonus marks for writing a new question and answering it on the exam

Yeah - known problem with the calculator, I predicted it before I even released it :P

The reason is because at the top end of the result curve, all the results bunch up and so it's hard to differentiate each student. The good news is this effect goes down REMARKABLY as you move away from 100%. For example, my calculator predicts that the difference between 100% and 95% is only 2 scores - most people would it expect it to be way more (for context, they say at 50 for further, 1 mark lost = 1 score dropped. 5% is 4 marks in exam 1, and 6 in exam 2). Then the difference between 90% and 95% is only 1.5 scores - which, again, it should be way more.

I think that the 90% scores aren't too much off the mark as a result, but yeah anything above that is DEFINITELY being underpredicted, especially for subjects like Further with more students clumped up around the top (i.e., smaller convergence to a bell curve).

I do wish I could fix this, or at least tell you how much the calculator is likely underpredicting you, but I can't do that without student's scores to test with, sorry :(
Title: Re: Study Score Calculator (open beta)
Post by: onlytherealonesknow on September 16, 2020, 07:13:08 am
Yeah - known problem with the calculator, I predicted it before I even released it :P

The reason is because at the top end of the result curve, all the results bunch up and so it's hard to differentiate each student. The good news is this effect goes down REMARKABLY as you move away from 100%. For example, my calculator predicts that the difference between 100% and 95% is only 2 scores - most people would it expect it to be way more (for context, they say at 50 for further, 1 mark lost = 1 score dropped. 5% is 4 marks in exam 1, and 6 in exam 2). Then the difference between 90% and 95% is only 1.5 scores - which, again, it should be way more.

I think that the 90% scores aren't too much off the mark as a result, but yeah anything above that is DEFINITELY being underpredicted, especially for subjects like Further with more students clumped up around the top (i.e., smaller convergence to a bell curve).

I do wish I could fix this, or at least tell you how much the calculator is likely underpredicting you, but I can't do that without student's scores to test with, sorry :(

All good just thought it was funny. :)
Title: Re: Study Score Calculator (open beta)
Post by: everythangcoZ on September 18, 2020, 03:41:11 pm
Hi, don't know if this question's been asked, but has this spreadsheet been updated to suit the new maths study design (methods)?
(the weightings have changed and GA1 total is now 75 from memory) or is it because it is based on last years graded distributions, that accurate estimations can't be made if it was adjusted?
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 18, 2020, 04:54:29 pm
Hi, don't know if this question's been asked, but has this spreadsheet been updated to suit the new maths study design (methods)?
(the weightings have changed and GA1 total is now 75 from memory) or is it because it is based on last years graded distributions, that accurate estimations can't be made if it was adjusted?

So, good question - the simple answer is that while the totals are from last year's study design, that's only because the grade distribution is from last year's study design, so I can't use this year's totals. The contribution of each GA, however, is from this study design, and so you can still use this calculator for this year's subjects.

The long answer:

The totals aren't used in the calculation of your actual score. The way this calculator works is it compares your percentage mark for each GA to the bell curve from last year, and places you on that bell curve. If you then multiply each of those positions by the percentage contribution, they then give you a placement in the bell-curve for the overall subject. This method works under the assumption that the upcoming bell-curve/grade distribution is going to be similar to last year's bell-curve/grade distribution - which seems like a bad assumption, however surprisingly, the grade distributions don't actually change all that much from each year to year.

If I don't use last year's totals, I can't compare to you last year's bell-curve, because your final score will be biased against you. However, unbiased results are preserved in linear combinations - and changing your score total to the new total is a linear transformation, and so by changing the total in this way, we preserve the estimate as an unbiased one.

And this is all the total is used for - it's not used to calculate the predicted score, ONLY to convert your current SAC score to one that can be compared to last year's distribution. Interestingly, the bigger potential cause for concern is that not all SACs are weighted equally, however the calculator is asking you for an overall percentage, and many are likely to just take the average of their SACs. Eg, let's say your Further scores are:

U3 Application task: 39/40
U3 Problem solving task: 15/20
U4 Problem solving task: 9/20

This would give a U3 SAC score of 54/60 (90%) and U4 SAC score of 9/20 (45%), equation to a total GA1 of 78.09%, but an average SAC score calculation would give score of 86.25% for unit 3 and 45% for unit 4, with a total GA1 of 75.33% - that's almost 3% difference! However, this difference actually decreases after you apply the SAC contribution - in fact, while the second method is technically under-estimating your study score (note: this isn't always going to happen. A low application task but high problem-solving task would have the opposite effect, and the more different the two scores, the more drastic the effect), the calculator predicts that a study score from a GA1 of 78.09% and 0 in everything else would be 16-21, and for a GA1 of 75.33% and 0 in everything else would 15-21. This is because the uncertainty of trying to predict your scores from last year's distribution is so much greater than the error in calculating the SAC scores wrong, that you basically don't notice the SAC score error in the final result.



Anyway, slight tangent at the end there, but hopefully the short-answer has answered you appropriately and you feel faith in still using the calculator.
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 21, 2020, 02:10:57 pm
Double-post, but it makes sense to put this thread back on the radar with an update this big, so I'ma do it and let the mods tear me a new one if they don't think it is okay



Thanks to another generous user, I now have another 6 subjects to play around and test with! They also did Japanese, which hasn't been requested yet but I wanted to test for still, so I added Japanese SL to the calculator. The results were... Interesting. The good news is, the calculator is essentially operating within precision - or at least to a level I'm satisfied with. So if you're not interested in the lengthy technical discussion, you can stop here

So, without revealing who this person is and what scores they got, they did quite well. As in, subjects with only A+ grades, quite well, though they unfortunately did not have their exact grades to test with - but this isn't that big a deal, and I'm more interested in just the letter grades for testing ANYWAY. And what I discovered is that the calculator is actually OVERestimating your score at the top end a little bit.

So for my previous test, I found that even if I took the extreme marks for the letter grades, I still got the person's study score within 20% of variation of the z-score - currently, the calculator uses a 25% variation buffer for the z-score. In this case, on the lower-side of the marks, they were within 20% - but on the higher-side (which essentially equated to all 100%s), the calculator was over-estimating the score, and I'd have to adjust the variation by up to 50% in some cases (this was for Japanese. Sidenote: Japanese, and presumably the other LOTEs as well, is really difficult for the calculator to predict!)

My hypothesis (but again, would require more scores and more testing - I don't think I'll ever have enough to be truly certain of what's going on, but just having as many scores as I can get is honestly a real boon) is that the calculator is actually a lot more robust that I gave it credit for, and that estimating scores up to about 44 is something it can do quite well in general. It's just the true extremes where it's going to struggle, and I think based on the subject we'll also see some major issues. In fact, I'd even go as far as to say that most subjects are fine with estimating scores up to even 48, and the only ones where we'll see some issues are subjects like Further which are particularly top-heavy in their distributions.

So, how do you tell if a subject is top-heavy or not? Well, what you want to do is look at the shape of the graph. We'll use the chemistry grade distribution as an example. Notice how GA2 (unit 4 SACs) has a median that's quite high (a B+), and a mode that's very close to the top (the mode simply being the highest dot), and the graph in general is very asymmetrical? That's top heavy. However, in GA1 (unit 3 SACs), the median is in the middle (a C+), the mode is also very in the middle, and the curve looks nice and symmetrical - this one isn't top heavy. So, you've expect the calculator is likely going to under-estimate for GA2, but calculate GA1 and GA3 fine (although the extreme end of these will be overestimated, as discussed above, but only the extreme end). Since GA1 and GA3 are worth 80% of your score, versus 20% for GA2, this ends up meaning that the calculation for chemistry is actually fine and quite good, even if overestimating at the very extremes.
Title: Re: Study Score Calculator (open beta)
Post by: Focused on September 23, 2020, 10:17:29 am
Can someone help me predict a study score for this year please?

SAC: 81% U3, 88% U4
Both SAC scores were in the upper range of the upper quartile. So I'm probably in the top 5 of a medium cohort.

Exam Scores Prediction:
E1: 35/40
E2: 65/80

I go to a private school but the cohort is not the strongest. For methods there are 39 people in my cohort sitting methods. I just need to know if I can get above 40 with this and hopefully if 45 is still in reach. Thank you so much! :)
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 23, 2020, 12:37:41 pm
Can someone help me predict a study score for this year please?

SAC: 81% U3, 88% U4
Both SAC scores were in the upper range of the upper quartile. So I'm probably in the top 5 of a medium cohort.

Exam Scores Prediction:
E1: 35/40
E2: 65/80

I go to a private school but the cohort is not the strongest. For methods there are 39 people in my cohort sitting methods. I just need to know if I can get above 40 with this and hopefully if 45 is still in reach. Thank you so much! :)

Bit of an odd place to ask since my calculator is meant to allow you to be able to calculate this for yourself. Anyway, I'm going to take the assumption that your SACs won't change, as they're fairly close to your exam marks and you're in the upper quartile of your cohort. I don't promise they won't change, but that's how I'm choosing to predict your score.

Doing this, my calculator predicts you to be in the range of a 37-41 - which does sound about right. Is a 45 in reach? Definitely not with those exam scores. If you can ace those exams (and I mean TRULY ace them), then maybe you can still get a 45 with a bit of luck, but you're definitely not on track as of right now for that to happen.
Title: Re: Study Score Calculator (open beta)
Post by: Annon4589 on September 29, 2020, 05:46:19 pm
I dont know if this is any help for your calculator, seeing that weighting is different this year, but these were my scores in psychology and religion and society:

Psychology
GA1: 95%
GA2: 97%
GA3: 81.25%
SS: 44

Religion & Society
GA1: 94%
GA2: 100%
GA3: 91.25%
SS: 48
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 30, 2020, 02:41:22 pm
I dont know if this is any help for your calculator, seeing that weighting is different this year, but these were my scores in psychology and religion and society:

Psychology
GA1: 95%
GA2: 97%
GA3: 81.25%
SS: 44

Religion & Society
GA1: 94%
GA2: 100%
GA3: 91.25%
SS: 48

You beautiful person, thank you very much. So, the interesting thing is, that the weighting makes VERY little difference in calculating the scores. With the old weighting, for example, my calculator predicts you'd get a 43.47 for R&S, but with the current ones, it predicts a 43.38. The new weightings have very little impact on things. Still worth updating them next year, but I can still use previous scores to benchmark what I've got.

---


Okay, so I think English Language and Religion and Society are the only two new subjects? Neat in its own way. As I feared, high scores are getting very underestimated. I need to find a way to up the variability more on higher scores while leaving lower scores not as affected, but I'm not quite sure how to do it. Will keep y'all updated.
Title: Re: Study Score Calculator (open beta)
Post by: Coolgalbornin03Lo on September 30, 2020, 03:46:20 pm
. As I feared, high scores are getting very underestimated.

But then people will be pleasantly surprised on results day!

Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on September 30, 2020, 04:53:16 pm
But then people will be pleasantly surprised on results day!



Yeah, I'm more worried about people that see a lower score and lose heart, when they could be well on their way to get a 45. Having the proviso and warning that higher scores get underestimated helps, but it'd still be nice to just have a better prediction.
Title: Re: Study Score Calculator (open beta)
Post by: homeworkisapotato on October 01, 2020, 11:39:13 am
Hey! If you're in a strong cohort, does your sac rank really matter for moderation?
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on October 01, 2020, 11:50:25 am
Hey! If you're in a strong cohort, does your sac rank really matter for moderation?

Absolutely - there are four data points VCAA use for SAC scaling:

-The top score (rank 1)
-The upper quartile (the top 25% of scores)
-The median (the top 50% of scores)
-The lower quartile (the top 75% of scores)

So, you can kind of think of this as the people in those positions gatekeeping your score. If you're in the top 33% of your class, and the person who is the top 25% gets a moderated SAC score of a B+, you CANNOT get higher than a B+. If your school is so strong that all of these people are getting an A+, and the SACs won't scale down, then it won't matter - but it's still in your interest to be as close to the top as you can get. Being in the top 25% should keep you safe. Eg, let's say you're in a class of 10, and the scores are (before moderation):

96%, 87%, 86%, 81%, 76%, 75%, 73%, 71%, 68%, 65%

Then the scores that will be used to moderate your cohort are:

96%
86%
76%
71%

So if you got that 81%, it's not going to scale higher than the person that got the 86%.
Title: Re: Study Score Calculator (open beta)
Post by: homeworkisapotato on October 01, 2020, 11:57:03 am
Thank you!!! So if in the exam I score higher than some ranked higher than me, it's not possible for me to have a higher scaled sac mark than them after moderation? So if a high ranked dude doesn't do well in the exam that'll decrease the maximum mark the rest of the top 25% can get?
Edit: I have heard of people not in the top 10-15 who have gotten above 45 so I was just curious :)
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on October 02, 2020, 12:12:19 am
Thank you!!! So if in the exam I score higher than some ranked higher than me, it's not possible for me to have a higher scaled sac mark than them after moderation? So if a high ranked dude doesn't do well in the exam that'll decrease the maximum mark the rest of the top 25% can get?
Edit: I have heard of people not in the top 10-15 who have gotten above 45 so I was just curious :)

No, you shouldn't walk away with a scaled SAC mark higher than them - VCAA very explicitly states that SAC moderation preserves the ranking of students. However, that doesn't mean you can't score higher than them on the exam - and don't forget, the closer your SAC score is to rank 1, the closer you are to getting scaled to similar SAC marks as them. So sure, in higher strength cohorts, you will get higher SAC marks scalings, making it easier to score a 40+ score, but your rank is still important - rank will be less important the closer your score is to rank 1, because even if you're the lowest rank, if your SAC mark is only 5% different to rank 1, you're probably getting scaled to a very similar mark as them ANYWAY.
Title: Re: Study Score Calculator (open beta)
Post by: Matthew Calabi on December 10, 2020, 12:46:23 pm
Hey, just a quick question, how accurate would the scores look if I scored low 80s on one exam, but high 90s on another? I heard about the whole underestimating thing for high scores, so do you think how would that come into play for skew between performance on both exams?
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on December 10, 2020, 01:21:53 pm
Hey, just a quick question, how accurate would the scores look if I scored low 80s on one exam, but high 90s on another? I heard about the whole underestimating thing for high scores, so do you think how would that come into play for skew between performance on both exams?

Good question - it depends on which subject, and how far bunched up the top end is in the grading distribution. The more bell-curvey that the grade distribution looks, the better the estimate. Let's use the methods study design as our case study. For GA1, the SACs, things aren't great. It starts off like a bell curve, but at the end it kind of flattens at 10% instead of going back to 0. But GA2 is even worse, and it only looks like half the bell-curve is there! As a result, higher scores in GA2 are going to lead to more underestimation than higher scores in GA1 will. GA3 looks the best, but again, it stops at 10% instead of continuing down to 0 - but it will still be better than GA1, since it still has the right shape.

However, your question asks if there's an issue with different scores on the exams - and there 100% is not, because the exam scores are used independently. So, as long as you're using the correct exam score in the right GA, then the scores accuracy will only be affected by a singular exam being too close to 100. Even if 1 exam is 20% and another is 80%, that difference isn't going to affect the accuracy in any way, shape, or form.
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on January 10, 2021, 01:31:35 pm
So, update - I think I can call the calculator a moderate success!

There are some scores it just cannot predict - for example, a Further study score of higher than 45 - and I'm not quite sure how to fix that (yet). However, it CAN accurately predict up to about 42 without any problems - but these problems are only with Further. Some other subjects, like methods, it seems to extremely accurately predict study scores, no matter how high or low they seem to be. This makes sense for a lot of mathsy reasons I'm not going to go into right now, but means I'll likely feel very comfortable potentially walking away from the project this year and let it self-manage.

Firstly, some disclaimers: this calculator only works if the grade distribution for the next year is similar to the grade distribution data it's using (right now, this is the data for 2019). However, this turnaround has shown that the distributions should be similar enough each year that you can still use the old data to numerically predict the new data. This makes sense, because that's how our current users predict study scores already. They use what they know about the subject, and what marks usually correlate to what scores, to estimate what study score you would get with the marks you have. For that reason, I believe that this calculator is about as accurate as any user on the website, with some small exceptions.

Right now, there's only one exception I've managed to find. That one is Further - if your GAs are all higher than 95%, then your score will be underestimated. Between 90% and 95% seems a slight grey-zone, where I'm not too sure. Most other subjects seem to be pretty well predicted, even on the higher end of things. Most importantly, the calculator is really good for scores between 20 and the low-mid 40s for every subject, which is where most people will be scoring.

If I had more time, I would design the calculator to use special error bounds for different subjects, so I could properly release the calculator - but I don't have the time for what was simply a pet project.

---

After all of that, simple conclusion - anybody doing year 12 in 2021, you can rest easy that this calculator seems to be an accurate predictor. So, if you really want to predict your study scores throughout the year and to see if you're on track, then you should be fine using this tool.

But also, please read the instructions. The constant e-mails requesting access got really annoying after a while - it might not seem like much on your end, but at one point I got like 10 e-mails all in one day, my phone actually vibrated so hard it yeeted itself off my desk.
Title: Re: Study Score Calculator (open beta)
Post by: Mackenzie Aps on January 12, 2021, 09:14:27 pm
I think the SS and ranking system was unfair this year for example, Legal Studies - with a school cohort of 20, there were only 5 that got a SS in the 40's, the rest were low 30's.  With SACS which were hard (from a top private school), and grades around B - B+, students still only managed to get a SS of low 30's.
Title: Re: Study Score Calculator (open beta)
Post by: BiggestVCESweat on January 13, 2021, 12:34:00 pm
I think the SS and ranking system was unfair this year for example, Legal Studies - with a school cohort of 20, there were only 5 that got a SS in the 40's, the rest were low 30's.  With SACS which were hard (from a top private school), and grades around B - B+, students still only managed to get a SS of low 30's.
On average, a school should expect 9% of the study scores to be above 40.
25% is a comparitively large amount, even for some of the best schools in Victoria.
Title: Re: Study Score Calculator (open beta)
Post by: keltingmeith on January 13, 2021, 01:08:56 pm
I think the SS and ranking system was unfair this year for example, Legal Studies - with a school cohort of 20, there were only 5 that got a SS in the 40's, the rest were low 30's.  With SACS which were hard (from a top private school), and grades around B - B+, students still only managed to get a SS of low 30's.

BiggestVCESweat already worded it really well, but more importantly - you talk about the ranking system as if there's some manipulative person behind the scenes deciding who should get each mark. That's not the case - it is literally a point of each student is turned into a numerical value, and then all of the numerical values are put on a number line to see who got the highest mark, second highest, third highest, etc., and the ranking figured out from there.

Maybe it feels unfair because your school didn't do as well as it normally does - that's not a point of the system being vindictive, the system inherently can't be vindictive because it's just following an algorithm. I'm really sorry, as well - maybe you didn't do as well as you would've liked to have done. But it's not the system's fault, sorry to say, and I'm sorry you feel frustrated by it :(
Title: Re: Study Score Calculator (open beta)
Post by: raghav0711 on March 07, 2021, 05:48:48 pm
Could you add environmental science and economics please thanks.
Title: Re: Study Score Calculator (open beta)
Post by: SOULXX on April 20, 2021, 07:37:32 pm
Hey Kelting,  can you please add  physical education onto the study score calculator.  Please
Title: Re: Study Score Calculator (open beta)
Post by: Mackenzie Aps on August 07, 2021, 09:38:30 pm
Hi can I get an estimate SS on results so far:
English: SAC's A+, A, A - I have my Oral SAC result still to come, hoping for A or A+.  Hope to score A/A+ for remainder of year including Exam
Methods: SAC - C on first, second one hope to get a B, and hope to finish year and exam with B/B+
Revolutions: SAC's for French rev - C and B.  Hope to get B for Russian Rev SAC's and estimate B+/A for exam
Chemistry: SAC's so far: B, A, A, B+ - hope to get B+/A's for rest of SAC's and estimate A for exam
French: SAC's so far: B, B+,B+, estimate I will get b/B+ for my Oral SAC and B+ for remainder SAC's, hope to get B+/A for exams
Legal Studies: I did in Year 11 2020 - not a great year with remote learning really affecting me - GA1 was B, GA2 B+ and GA3 B+ = 33SS