Schools are not allowed to reject enrolment from a subject for individuals. However as we know, there are many reasons or strategies schools use to do this within the lines e.g. not run the subject, the "encouragement" one that PF stated is one i've seen many times.
At my previous school I worked at, we were told that decisions relating to subject enrolment must be up to the student and parent/guardian. As a school they should only be permitted to make recommendations. Of course, if a subject can't run due to lack of numbers etc. (particularly in the public system, this is quite common and an acceptable reason) then this is a different story. But individual rejection or withdrawal due to impact on scores is wrong and should not technically happen - unless it is a VCAA pre-requisite which is embedded in the study design (e.g. Methods 1/2 for Algorithmics). Timetabling is the only genuine reason I can think of that would be an appropriate reason for rejecting enrolment. If a subject can't run due to lack of enrolments, fair enough. If you can't enrol in it because your other choices conflict with the timetable, fair enough (can't cater to everybody, timetabling is such a bloody complex process... respect to all timetablers out there).
Decisions relating to program changes (e.g. VCE to VCAL, vice-versa, etc) should DEFINITELY be done with permission from the parents... infact any decision relating to subject changes should be done with parent permission anyway given the potential impact it has on future university prospects. At our school any subject change decision (any year level) requires both student and parent authorisation before it happens (usually done through a signed form or meeting).