Except it isn't, is it?
I've traveled on dozens of trains back from the football and on every single one my experience has been people sat in their seats and not singing. Maybe a little bit of chatter that is slightly louder than normal, but certainly not sweary or intimidating others.
In fact, I'd wager the majority of people here have traveled on "football" services without realising it, because the fans have kept themselves to themselves and not caused an issue. If it's only 4 or 5 trains a weekend then that's clearly the minority (46 games a weekend, probably multiple services going there and back), but you get that with EVERY event or large group.
If the media didn't jump on the bandwagon of criticising football fans every time anything happened (as opposed to rugby, which many guards/stewards/police claim is MUCH worse than football, or racing, stag dos, hen dos, cricket etc.) then I imagine most people would consider it a bit of fun and let them get on with it.
There are many many reports available showing the arrest numbers at football are far lower than nearly every other major event (all the major music festivals, Notting Hill carnival, and so on), and given how heavy-handed policing is at football (I've known people to get arrested for throwing their bottle of water to the ground in frustration at football), that's pretty astonishing. Yes there are a small number of idiots who ruin it, and a slightly larger number who shout and get a bit lairy, but they'd do it at some other event if you cut it out for football.
The problem with public transport is that you have to share it with the public. Doesn't make their behaviour right, but you need to accept that it's not going to a library!