It's not necessarily that Literature is hard to grasp, it's more that I think you need to be willing to work at getting your mind around the differences between VCE Literature and VCE English. I didn't really do much in 1/2 Lit, so I found 3/4 Lit was a steep learning curve for me and caused me a great deal of anxiety in first term.
But I think persistence really helped me out, because now it is one of my favourite subjects. It can be considered more difficult to score well in Literature because you are against the top writers in the state, compared to those in mainstream English. But like any subject, the more practice and time you put into the subject will help you to achieve your best, so I'd say if you want to take a challenging but highly regarding subject, then take Lit.
Understanding how to go about writing a passage analysis for Lit can be difficult at first, but once you understand this and work out a way that works best for you, it's not really that bad. As I've said on this forum before, English is more theme based but Lit is more language based. I've also found that the skills I've developed in Lit have immensely helped my English skills. Would taking both Lit and English be an option?