I didn't do VCE physics and was in a similar situation to you. Therefore, I decided to pick Physics 1 Fundamentals and Physics 2 Life Sciences (which, I think, is more targeted towards health sciences).
Personally, I found both semesters challenging, but very interesting. Physics fundamentals was a bit of a shock: fast paced and a completely different way of thinking to other sciences I have done (chem + bio). The bad lecturers didn't help either. Specialist maths, however, will give you an advantage
In comparison, the lecturers in second semester were fantastic. The content was more interesting and example based (hence more difficult).
Although I received better marks in first semester, I definitely found second semester more enjoyable.
Even without VCE physics, it IS possible to get H2As and H1s with a bit of effort and study.
I would recommend completing both semesters of physics, since not all of the VCE material (hence the stuff that will be tested on the GAMSAT), is taught in fundamentals.