I've only done English Language, so I can't really compare the two, but I'll tell you what I think about the subject.
The content in English Language 3/4 really isn't extremely interesting (unless you're keen on linguistics), but I found that I didn't have to study that much for it because it was mostly analytical stuff that came naturally to me. I personally feel like the workload in mainstream English seems rather more painful and tiring from what I've heard, but it all depends on the type of person you are and what you're more comfortable doing, I guess.
Generally speaking, maths/science students are better suited to English Language, but everyone's different, so it's really hard to judge. You should still be able to do well in English Language without having done 1/2, especially if you're into critical analysis. But if you're into more broader thinking, then by all means stick with regular English.