Hey I'll be able to provide some insights into this as I'm one of those who didn't take VCE Physics but was required to in uni. Take it with a grain of salt as this is 4 years ago so there will definitely be things that had changed. (The old physics labs were demolished, and is replaced with a much nicer learning environment. My friend tutors first year physics and he said that the labs are currently alot more cooperative instead of in pairs back in our year. Apart from that the content is roughly the same)
Alright so I'll start with foundation physics (PHS1080/ENG1080). This is a combined sci/eng unit where both faculty will share the unit. Personally, I found this unit to be easy and pretty straightforward if you have a solid mathematical background. Specialist maths would help quite abit if you did it in VCE. (From your sig, it appears that you didn't but don't worry they start from the absolute fundamentals). This is your honeymoon physics unit.
With the first semester of physics down, you've probably scored reasonably well in foundation physics and think that you're decent in it. Then comes semester 2 (PHS1022) where you'll be together with the rest of the students which had completed VCE physics and PHS1011 (A harder version of PHS1080). This is when you'll feel the gap in your knowledge. The lecturers will frequently assume a fair bit of previous knowledge so I'll recommend that you read ahead before every single lecture. In this unit, your level of interest in physics will start to show in your progress. The mid semester tests will be brutal and will be the first time a fair number of students get a horrible mark in a test. At the end of this unit, you should be at par with the rest of the cohort who've done VCE physics. You'll just need to work much harder during this semester to fill up the gaps in your knowledge. This unit will be hard but still doable if you're just moderately interested in physics.
Moving onto second year physics, this is where you feel the heat if you're not into physics. I won't go into much detail as there's a few unit review in the relevant thread which you can read up on. I dropped my the science component of my double degree because of second year, second semester physics (PHS2022) when I realized that I started dreading uni because of physics but thats down to each individual.
Anyway in summary, just take up foundation physics (PHS1080), try and understand and appreciate the concepts. Then work really really hard in semester two (PHS1022) and at the end of first year you'll be on the same level as others. From there on, decide on wherever you want to go.
Also, welcome to the hood.