Honestly, I don't see any reason why you shouldn't give it a try. No doubt that it will be hard though; even for people who have never skipped a year of Vietnamese find VCE Vietnamese hard, but I honestly think you should try Unit 2 Viet (which is the same as Unit 1 really) and decide whether to pursue Vietnamese in year 12 after you finish Unit 2.
Keep in mind that that for people, born in Australia, that it is writing that is the most difficult, so make sure to concentrate on improving your writing as this will be an important factor in doing well for this subject. Read Vietnamese newspaper and look at the vocabulary and spelling of harder words. Go through the Viet textbooks thoroughly and memorise the spelling and translation of harder vocab as a good vocab will generally allow you to do well. To improve speech and pronunciation, regularly talk to your parents in Vietnamese (if they are Viet of course!) and listen to their criticisms and take them on board.
Also check out my guide for general tips and tricks and sample essays to start preparing for Unit 3 and 4 Vietnamese!
Wish you luck!