I think that you should do Chem 3/4 if your school allows you to do so. Scraping a 25 (minimum pre-req score) is fairly doable with some effort. While it may bring down your ATAR a bit as you may have to drop a subject may do better at (even though it will probably counted as a 10% subject so it will hurt less), a high ATAR is only so useful insofar as it allows you pursue courses and pathways that lead to your preferred career. There are quite a few humanities inclined students (as well as mathematically inclined ones who aren't strong in Science), that did Chemistry instead of a subject that they were stronger in so that that could pursue med (or science based courses that can lead to med). If you are really set on Med, a slightly lower ATAR that allows you to pursue that career is worth it. Even if you don't get into undergrad med (since some people find the UCAT difficult and the interviews are competitive), doing a science subject in year 12 is good preparation for university biology/chemistry subjects. It can be quite tough going from no science subjects for a couple of years to a few university science subjects. If you were to do a Bachelor of Arts in Unimelb (which could lead to a graduate med degree, I mentioned Arts because you wouldn't be able to do a Science degree since you don't seem to be planning on completing Methods, although you can transfer), VCE Chemistry would be helpful as there a limited number of breadth subjects for you to complete the pre-req subjects for Medicine. Not doing VCE Chemistry may potentially force you do an extra chemistry subject.
The Bachelor of Science at Monash is also another option (Further maths will suffice as a pre-req) as well as various health sciences and biomedical sciences in other Victorian universities (can't tell you the exact pre-reqs as I never looked into them when I was in high school, although I wish I did). In terms of graduate med entry; Deakin, ANU, Uni of Sydney, Griffith, UWA (Western Australia), UQ (Queensland), UOW (Wollongong) and Notre Dame do not require any science subjects at undergrad or high school level, although it would be advisable to complete a few as the GAMSAT (graduate entry exam) contains quite a lot of scientific content. University of Melbourne also allows you to complete an undergrad degree in any discipline but you have to complete a few Biology/Chemistry pre-req subjects.