I think there is a valid argument - Allowing gay marriage can alter the structure of family. Does being raised by two dads or two mums have any bad consequences? I don't know if it does or not, but there's a point there. Should we then not allow gay married couples to have access to things like IVF? Or would it be better to allow only adoptions (2 dads is better than no dads). I haven't given these points enough thought to formulate an opinion, but on the face of it, i think they constitute valid points
Good, probably the first point meriting detailed attention

Well, firstly, there's the issue that in many jurisdictions, marriage and adoption are completely two different systems legally. As far as I know, it is entirely possible for a de-facto gay couple in Australia to adopt. In California, same-sex couples have the exact same adoption rights as straight couples. So marriage does not necessarily equate to adoption rights.
Secondly, there is certainly a strong argument to let same-sex couples adopt. Even if a same-sex household is not ideal (questionable, see below), then they would almost certainly still be in a better situation than they were in before.
Thirdly, we allow single parents to adopt. Is that better for the child than a same-sex couple adopting them?
There is no evidence whatsoever which suggests that children raised by same-sex couples are at a disadvantaged position, except due to the bigotry of those around them. They do not suffer from mental health problems at an any greater rate, make just as much or even more money than their peers, and are educated to the same degree as well.