I wouldn't think so. If you make it thought the course you should be able to find job reasonably easily as there is a steady demand for actuaries (as russ said). Having extra mathematical knowledge may help, but like, a good interview would probably count for a lot more e.g. it's not uncommon at all for people who only did the act stud major to get jobs over honours students.
Having said that, i do plan to do a diploma myself, the diploma of music (practical) to be specific becuase not only do i really enjoy music and would like a few letters after my name to show the work that i have done over the years, but ECs like music are good for CVs

Also the big plus side is that even with the diploma, it will only take me three years to finish the degree.
But anyway, if i were in your position, i would stick to the main degree
