I did UMEP physics back in the day, so feel free to ask me questions. You definitely do NOT need specialist for the course; there were two of us out of 6 who were not taking it, and I got a HD, and I think he got a D. Just make sure you can integrate.
Also, don't base your interest in physics off VCE physics. Once you get out of VCE physics, you can finally start using calculus, although the math in UMEP certainly isn't difficult. You can probably ask the teacher if you need help.
If the prereqs haven't changed, you need a 40 in methods and a 40 in physics in year 11 to take the subject, and otherwise, you have to sit a test. My friends said the test was pretty easy - I got around it another way by having a High Distinction in the National Qualifying Exam for the Physics Olympiad, which is definitely something you should consider taking if you're interested in physics. In my year, Melbourne asked for photocopies of any olympiad results, so yes, it's a valid way of circumventing the requirements.