I really hate saying this, but whilst they aren't shit, the ranking of ANU is not as a result of its bad undergraduate programs.
ANU is either probably, likely or certainly the best research university in Australia. Quite a solid amount of funding goes into it and it pays off really well. But this, virtually in itself, is what makes it the top Go8 university. If you want to do research, transferring to ANU sounds like a great plan. However, this is definitely not necessarily the case for taking your undergraduate degree.
__________________
Now, between USyd and UNSW. USyd is known to have a strong maths department. If you choose just that single degree, and choose a maths major, you may as well go with the former.
As for the plethora of other majors, well there is a stereotype. That is, "UNSW is known for its STEM" - science, technology, engineering, mathematics. For science, I'd say it is quite true. At first glance, the chemistry labs look 'upgraded' with loads of apparatus there for you to use than what you'd expect. Feedback has been mostly good from many advanced science students that I've teamed up with. In particular, the biologists REALLY enjoy their stuff there, but I forgot why...
(Interesting how mathematics should be placed there as well. Indeed, UNSW's mathematics department is also excellent; it's part of why I study there. But UNSW is mostly known for a solid coverage across the entire spectrum; maths in itself is just one specialty.)
However, keep in mind that I did not compare and contrast to USyd just now - I am merely talking about UNSW itself. Some things to compare may be the fact that both physics buildings can be a bit difficult to 'live' in (buildings are somewhat old). UNSW have two courses known as SCIF1121 and SCIF1131 to which you must take one or the other. They were a bit boring... so if the courses you take matter then you might wanna see if USyd has something similar.
I wouldn't know much about USyd myself to leave opinions on it though