I haven't completed 3/4 physics, but I did do unit 1 and have been at DECV for a while. Across the board at DECV, especially the in the sciences, they are really weak cohorts. While this does make it harder to achieve a high score, it is not impossible. If you are rank 1 in your subject (maintain the highest SAC scores), and score high on the exam, then it is 100% possible to get a high score in physics.
Another thing I think should be pointed out, is how you find the course. I personally didn't like how the information was presented (especially introducing ohms law in week 3 of the subject instead of 1), and I didn't find the teacher very helpful, which made it a harder subject to study in the end. For instance, there weren't enough maths examples, so when it came time to the quiz's and SACs, I was quite unprepared. So if you think learning in a class environment would work better for you to achieve the results you want, keep that in mind. I would say the SACs are on the easier side at DECV, so they are at risk of scaling down.
There have been a few students over the years that have gotten high scores in the sciences, but it is very rare, and certainly not impossible. If you work hard and get the highest results in SACs + the exam, you should be fine to get the score you want. If you are an average student however, it will certainly be harder. As Bri said, talk to your teachers if you are not sure, and make a decision that works best for you.