Login

Welcome, Guest. Please login or register.

March 30, 2024, 02:14:45 am

Author Topic: Study Score Calculator (open beta)  (Read 44494 times)  Share 

0 Members and 1 Guest are viewing this topic.

everythangcoZ

  • Adventurer
  • *
  • Posts: 24
  • Respect: +1
Re: Study Score Calculator (open beta)
« Reply #15 on: September 18, 2020, 03:41:11 pm »
0
Hi, don't know if this question's been asked, but has this spreadsheet been updated to suit the new maths study design (methods)?
(the weightings have changed and GA1 total is now 75 from memory) or is it because it is based on last years graded distributions, that accurate estimations can't be made if it was adjusted?
2019|Biology
2020|Methods, Further, Psychology, Chemistry, English

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #16 on: September 18, 2020, 04:54:29 pm »
+5
Hi, don't know if this question's been asked, but has this spreadsheet been updated to suit the new maths study design (methods)?
(the weightings have changed and GA1 total is now 75 from memory) or is it because it is based on last years graded distributions, that accurate estimations can't be made if it was adjusted?

So, good question - the simple answer is that while the totals are from last year's study design, that's only because the grade distribution is from last year's study design, so I can't use this year's totals. The contribution of each GA, however, is from this study design, and so you can still use this calculator for this year's subjects.

The long answer:

The totals aren't used in the calculation of your actual score. The way this calculator works is it compares your percentage mark for each GA to the bell curve from last year, and places you on that bell curve. If you then multiply each of those positions by the percentage contribution, they then give you a placement in the bell-curve for the overall subject. This method works under the assumption that the upcoming bell-curve/grade distribution is going to be similar to last year's bell-curve/grade distribution - which seems like a bad assumption, however surprisingly, the grade distributions don't actually change all that much from each year to year.

If I don't use last year's totals, I can't compare to you last year's bell-curve, because your final score will be biased against you. However, unbiased results are preserved in linear combinations - and changing your score total to the new total is a linear transformation, and so by changing the total in this way, we preserve the estimate as an unbiased one.

And this is all the total is used for - it's not used to calculate the predicted score, ONLY to convert your current SAC score to one that can be compared to last year's distribution. Interestingly, the bigger potential cause for concern is that not all SACs are weighted equally, however the calculator is asking you for an overall percentage, and many are likely to just take the average of their SACs. Eg, let's say your Further scores are:

U3 Application task: 39/40
U3 Problem solving task: 15/20
U4 Problem solving task: 9/20

This would give a U3 SAC score of 54/60 (90%) and U4 SAC score of 9/20 (45%), equation to a total GA1 of 78.09%, but an average SAC score calculation would give score of 86.25% for unit 3 and 45% for unit 4, with a total GA1 of 75.33% - that's almost 3% difference! However, this difference actually decreases after you apply the SAC contribution - in fact, while the second method is technically under-estimating your study score (note: this isn't always going to happen. A low application task but high problem-solving task would have the opposite effect, and the more different the two scores, the more drastic the effect), the calculator predicts that a study score from a GA1 of 78.09% and 0 in everything else would be 16-21, and for a GA1 of 75.33% and 0 in everything else would 15-21. This is because the uncertainty of trying to predict your scores from last year's distribution is so much greater than the error in calculating the SAC scores wrong, that you basically don't notice the SAC score error in the final result.



Anyway, slight tangent at the end there, but hopefully the short-answer has answered you appropriately and you feel faith in still using the calculator.

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #17 on: September 21, 2020, 02:10:57 pm »
+4
Double-post, but it makes sense to put this thread back on the radar with an update this big, so I'ma do it and let the mods tear me a new one if they don't think it is okay



Thanks to another generous user, I now have another 6 subjects to play around and test with! They also did Japanese, which hasn't been requested yet but I wanted to test for still, so I added Japanese SL to the calculator. The results were... Interesting. The good news is, the calculator is essentially operating within precision - or at least to a level I'm satisfied with. So if you're not interested in the lengthy technical discussion, you can stop here

So, without revealing who this person is and what scores they got, they did quite well. As in, subjects with only A+ grades, quite well, though they unfortunately did not have their exact grades to test with - but this isn't that big a deal, and I'm more interested in just the letter grades for testing ANYWAY. And what I discovered is that the calculator is actually OVERestimating your score at the top end a little bit.

So for my previous test, I found that even if I took the extreme marks for the letter grades, I still got the person's study score within 20% of variation of the z-score - currently, the calculator uses a 25% variation buffer for the z-score. In this case, on the lower-side of the marks, they were within 20% - but on the higher-side (which essentially equated to all 100%s), the calculator was over-estimating the score, and I'd have to adjust the variation by up to 50% in some cases (this was for Japanese. Sidenote: Japanese, and presumably the other LOTEs as well, is really difficult for the calculator to predict!)

My hypothesis (but again, would require more scores and more testing - I don't think I'll ever have enough to be truly certain of what's going on, but just having as many scores as I can get is honestly a real boon) is that the calculator is actually a lot more robust that I gave it credit for, and that estimating scores up to about 44 is something it can do quite well in general. It's just the true extremes where it's going to struggle, and I think based on the subject we'll also see some major issues. In fact, I'd even go as far as to say that most subjects are fine with estimating scores up to even 48, and the only ones where we'll see some issues are subjects like Further which are particularly top-heavy in their distributions.

So, how do you tell if a subject is top-heavy or not? Well, what you want to do is look at the shape of the graph. We'll use the chemistry grade distribution as an example. Notice how GA2 (unit 4 SACs) has a median that's quite high (a B+), and a mode that's very close to the top (the mode simply being the highest dot), and the graph in general is very asymmetrical? That's top heavy. However, in GA1 (unit 3 SACs), the median is in the middle (a C+), the mode is also very in the middle, and the curve looks nice and symmetrical - this one isn't top heavy. So, you've expect the calculator is likely going to under-estimate for GA2, but calculate GA1 and GA3 fine (although the extreme end of these will be overestimated, as discussed above, but only the extreme end). Since GA1 and GA3 are worth 80% of your score, versus 20% for GA2, this ends up meaning that the calculation for chemistry is actually fine and quite good, even if overestimating at the very extremes.

Focused

  • Fresh Poster
  • *
  • Posts: 3
  • Respect: 0
Re: Study Score Calculator (open beta)
« Reply #18 on: September 23, 2020, 10:17:29 am »
0
Can someone help me predict a study score for this year please?

SAC: 81% U3, 88% U4
Both SAC scores were in the upper range of the upper quartile. So I'm probably in the top 5 of a medium cohort.

Exam Scores Prediction:
E1: 35/40
E2: 65/80

I go to a private school but the cohort is not the strongest. For methods there are 39 people in my cohort sitting methods. I just need to know if I can get above 40 with this and hopefully if 45 is still in reach. Thank you so much! :)

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #19 on: September 23, 2020, 12:37:41 pm »
0
Can someone help me predict a study score for this year please?

SAC: 81% U3, 88% U4
Both SAC scores were in the upper range of the upper quartile. So I'm probably in the top 5 of a medium cohort.

Exam Scores Prediction:
E1: 35/40
E2: 65/80

I go to a private school but the cohort is not the strongest. For methods there are 39 people in my cohort sitting methods. I just need to know if I can get above 40 with this and hopefully if 45 is still in reach. Thank you so much! :)

Bit of an odd place to ask since my calculator is meant to allow you to be able to calculate this for yourself. Anyway, I'm going to take the assumption that your SACs won't change, as they're fairly close to your exam marks and you're in the upper quartile of your cohort. I don't promise they won't change, but that's how I'm choosing to predict your score.

Doing this, my calculator predicts you to be in the range of a 37-41 - which does sound about right. Is a 45 in reach? Definitely not with those exam scores. If you can ace those exams (and I mean TRULY ace them), then maybe you can still get a 45 with a bit of luck, but you're definitely not on track as of right now for that to happen.

Annon4589

  • Adventurer
  • *
  • Posts: 22
  • Respect: +1
Re: Study Score Calculator (open beta)
« Reply #20 on: September 29, 2020, 05:46:19 pm »
+5
I dont know if this is any help for your calculator, seeing that weighting is different this year, but these were my scores in psychology and religion and society:

Psychology
GA1: 95%
GA2: 97%
GA3: 81.25%
SS: 44

Religion & Society
GA1: 94%
GA2: 100%
GA3: 91.25%
SS: 48

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #21 on: September 30, 2020, 02:41:22 pm »
+5
I dont know if this is any help for your calculator, seeing that weighting is different this year, but these were my scores in psychology and religion and society:

Psychology
GA1: 95%
GA2: 97%
GA3: 81.25%
SS: 44

Religion & Society
GA1: 94%
GA2: 100%
GA3: 91.25%
SS: 48

You beautiful person, thank you very much. So, the interesting thing is, that the weighting makes VERY little difference in calculating the scores. With the old weighting, for example, my calculator predicts you'd get a 43.47 for R&S, but with the current ones, it predicts a 43.38. The new weightings have very little impact on things. Still worth updating them next year, but I can still use previous scores to benchmark what I've got.

---


Okay, so I think English Language and Religion and Society are the only two new subjects? Neat in its own way. As I feared, high scores are getting very underestimated. I need to find a way to up the variability more on higher scores while leaving lower scores not as affected, but I'm not quite sure how to do it. Will keep y'all updated.

Coolgalbornin03Lo

  • Forum Leader
  • ****
  • Posts: 542
  • Respect: +132
Re: Study Score Calculator (open beta)
« Reply #22 on: September 30, 2020, 03:46:20 pm »
0
. As I feared, high scores are getting very underestimated.

But then people will be pleasantly surprised on results day!

My avatar sums up life.
“I’m free to be the greatest one alive” ~ Sia
╔══════════════════════════════╗
2020: English | Methods | Biology | Chemistry |              Psychology | ATAR: 0
╚══════════════════════════════╝

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #23 on: September 30, 2020, 04:53:16 pm »
+5
But then people will be pleasantly surprised on results day!



Yeah, I'm more worried about people that see a lower score and lose heart, when they could be well on their way to get a 45. Having the proviso and warning that higher scores get underestimated helps, but it'd still be nice to just have a better prediction.

homeworkisapotato

  • MOTM: Aug 20
  • Forum Leader
  • ****
  • Posts: 527
  • Respect: +288
Re: Study Score Calculator (open beta)
« Reply #24 on: October 01, 2020, 11:39:13 am »
0
Hey! If you're in a strong cohort, does your sac rank really matter for moderation?
2020: Biology [43]
2021: Methods, Chemistry, HHD, English, Further
Selective School Entry Tips

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #25 on: October 01, 2020, 11:50:25 am »
+5
Hey! If you're in a strong cohort, does your sac rank really matter for moderation?

Absolutely - there are four data points VCAA use for SAC scaling:

-The top score (rank 1)
-The upper quartile (the top 25% of scores)
-The median (the top 50% of scores)
-The lower quartile (the top 75% of scores)

So, you can kind of think of this as the people in those positions gatekeeping your score. If you're in the top 33% of your class, and the person who is the top 25% gets a moderated SAC score of a B+, you CANNOT get higher than a B+. If your school is so strong that all of these people are getting an A+, and the SACs won't scale down, then it won't matter - but it's still in your interest to be as close to the top as you can get. Being in the top 25% should keep you safe. Eg, let's say you're in a class of 10, and the scores are (before moderation):

96%, 87%, 86%, 81%, 76%, 75%, 73%, 71%, 68%, 65%

Then the scores that will be used to moderate your cohort are:

96%
86%
76%
71%

So if you got that 81%, it's not going to scale higher than the person that got the 86%.

homeworkisapotato

  • MOTM: Aug 20
  • Forum Leader
  • ****
  • Posts: 527
  • Respect: +288
Re: Study Score Calculator (open beta)
« Reply #26 on: October 01, 2020, 11:57:03 am »
0
Thank you!!! So if in the exam I score higher than some ranked higher than me, it's not possible for me to have a higher scaled sac mark than them after moderation? So if a high ranked dude doesn't do well in the exam that'll decrease the maximum mark the rest of the top 25% can get?
Edit: I have heard of people not in the top 10-15 who have gotten above 45 so I was just curious :)
« Last Edit: October 01, 2020, 11:59:01 am by homeworkisapotato »
2020: Biology [43]
2021: Methods, Chemistry, HHD, English, Further
Selective School Entry Tips

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #27 on: October 02, 2020, 12:12:19 am »
+3
Thank you!!! So if in the exam I score higher than some ranked higher than me, it's not possible for me to have a higher scaled sac mark than them after moderation? So if a high ranked dude doesn't do well in the exam that'll decrease the maximum mark the rest of the top 25% can get?
Edit: I have heard of people not in the top 10-15 who have gotten above 45 so I was just curious :)

No, you shouldn't walk away with a scaled SAC mark higher than them - VCAA very explicitly states that SAC moderation preserves the ranking of students. However, that doesn't mean you can't score higher than them on the exam - and don't forget, the closer your SAC score is to rank 1, the closer you are to getting scaled to similar SAC marks as them. So sure, in higher strength cohorts, you will get higher SAC marks scalings, making it easier to score a 40+ score, but your rank is still important - rank will be less important the closer your score is to rank 1, because even if you're the lowest rank, if your SAC mark is only 5% different to rank 1, you're probably getting scaled to a very similar mark as them ANYWAY.

Matthew Calabi

  • Fresh Poster
  • *
  • Posts: 2
  • Respect: 0
Re: Study Score Calculator (open beta)
« Reply #28 on: December 10, 2020, 12:46:23 pm »
0
Hey, just a quick question, how accurate would the scores look if I scored low 80s on one exam, but high 90s on another? I heard about the whole underestimating thing for high scores, so do you think how would that come into play for skew between performance on both exams?

keltingmeith

  • Honorary Moderator
  • Great Wonder of ATAR Notes
  • *******
  • Posts: 5493
  • he/him - they is also fine
  • Respect: +1292
Re: Study Score Calculator (open beta)
« Reply #29 on: December 10, 2020, 01:21:53 pm »
0
Hey, just a quick question, how accurate would the scores look if I scored low 80s on one exam, but high 90s on another? I heard about the whole underestimating thing for high scores, so do you think how would that come into play for skew between performance on both exams?

Good question - it depends on which subject, and how far bunched up the top end is in the grading distribution. The more bell-curvey that the grade distribution looks, the better the estimate. Let's use the methods study design as our case study. For GA1, the SACs, things aren't great. It starts off like a bell curve, but at the end it kind of flattens at 10% instead of going back to 0. But GA2 is even worse, and it only looks like half the bell-curve is there! As a result, higher scores in GA2 are going to lead to more underestimation than higher scores in GA1 will. GA3 looks the best, but again, it stops at 10% instead of continuing down to 0 - but it will still be better than GA1, since it still has the right shape.

However, your question asks if there's an issue with different scores on the exams - and there 100% is not, because the exam scores are used independently. So, as long as you're using the correct exam score in the right GA, then the scores accuracy will only be affected by a singular exam being too close to 100. Even if 1 exam is 20% and another is 80%, that difference isn't going to affect the accuracy in any way, shape, or form.