Spiral binding sounds a good idea, it'd make it easy to detach the tests if you really wanted to as well. I don't think it'd look too bad, I'm guessing that stripey bit on the cover design would be extended a bit etc.
It would mean people can't use your books as a bound reference, if they are permitted one.
Page 6, bound reference rules
http://www.vcaa.vic.edu.au/Documents/vceexamnav12.pdf• the form of binding is not specified but it must be secure, and pages must not be readily detachable or designed to be removed. Binding can include cloth, glue, staple, spiral or comb binding
Pages are kind of detachable, but they've explicitly said it's okay.
If the maths books are similar to the chem book, I imagine it'd be awkward to use the book as a bound reference though. I guess it'd be similar to using the textbook as a bound reference though, which is fairly common.