I would just simply say "drop chem because you don't enjoy it", but obviously, a lot of the medical/science courses at Unis require VCE Chemistry to be completed, that doesn't work. Chem 3/4 does indeed get a *little* harder, but probably not to the same extent how Specialist 3/4 gets harder. So, sadly, despite the fact that you don't enjoy it too much, if you really want to get into Biomedicine, you're going to have to stick with it. Hopefully there is a way you can establish stability in your grades whilst also enjoying the content.
Psych 3/4 does not require 1/2, except for research methods. If you don't have the opportunity to learn and get close to mastery with research methods, it doesn't make 3/4 impossible to do well, but it would require a fair extra amount of revision and study to compete with those who learned it in 1/2. Other than that, it's pretty much new content that doesn't rely on the previous two units.