It depends on how much you enjoy Specialist in Year 11 or Maths throughout your schooling. Although I know what pi is getting to, it's hard to predict whether you can get a 38 or 39 or above because of circumstantial changes. Also what you think you can get in another subject does not equate the same score in Specialist because of cohort strength - for instance, in an average methods exam 2 10-15 marks can still give you an A+, in Spesh this can range from 5-10 In exam 1, 3-7 marks can get you an A+, in past two years you could only afford to drop 1.5-2 marks for A+ in Specialist.
I, too, thought I would get a 38+ for Spesh but ended getting 33 (45) and that didn't even make my top four. And worse still, I spent the most time in Specialist (legit) but due to poor exam performances - and any subject that doesn't make top four is wasted. My SACs were scaled down accordingly due to my crappy exam performance (so no MHS sac guarantees).
So, are you (I hate using this) a "gifted" maths student? For me, it showed that hard work perhaps won't be enough to reach the upper echelon of Spesh (i.e.. 50 or above scaled). So, definitely give PE a consideration and don't rely 100% on scaling (although it does help a bit).