Definitely depends on who you're hanging out with, but I am curious - is it specifically about drinking that the issue lies in, or is it not wanting to go out to a pub?
Because I can understand the whole going to drinks after cricket, because I see it all the time at conferences. After a full day/session/whatever of work, everyone wants to loosen up and unwind. So, we all go to the pub. Not everyone drinks, we don't make anyone feel bad if they're not (although we sometimes make a show if it's someone we know is usually up for it and isn't this time - potentially toxic culture there, but I also hope that if this was an issue, anyone we did it to would speak up about it, given that nobody has yet), but for those of us that do want to, the pub is basically the only option for us. Given the drinkers normally outweigh the non-drinkers, we end up there, so I can see why you might feel left out if that was the case, but I don't think you necessarily are being left out?