Hi Flyboy
I have no doubt you could do the job, however I do think you under estimate the advantage railway experience can give you sometimes. You say the guard and station customer advisor got to the second stage, maybe this gave them the edge has they had railway experience in using the rule book and could demonstrate more relevant experience in their application. I'm not saying it will get you the job or guarantee shortlisting, but it may help. I was previously a signaller and have got a trainee train driver job starting in Nov. There were several others at my assessment day who had current or previous rail experience.
The other thing you mention is that you have only ever applied to one TOC officially via an application, so you can hardly base one failed application on your skills/experience versus someone else's. You need to get out there and keep applying via the official channels. Don't bother sending speculative CV's, especially for TOCS of FOCS that very rarely take any trainees on, use the applications when the jobs arise, no matter how tedious they can be.
I have applied to WAGN (2004) and four this year - First Scotrail, Northern, First Capital Connect and Greater Anglia. A failed the first driver assessment on the dots in 2004, was a Signaller for Network Rail from 2007-2008, this year got to the online assessments for Northern, shortlisted and on hold for the First ones and passed everything for GA (which is the closest to where I live after all that). All the TOCS have different criteria and some either like your application and some don't. I have used my basic application throughout, with tweaks to fit each application and have been successful in the end. But you just need to apply more and I have no doubt you'll find a TOC that will shortlist you and then the ball's in your court. It's not always a quick fix!
At the end of the day, I do believe some element of it is luck. With so many applications being received, so many TOCS with different criteria and different recruitment officers with their own views/opinions. But those elements are out of your hands, so just try a few more proper applications. If you want it, you have to put the work in and can't afford to become complacent. You get the knock backs, but just keep applying. Keep us posted!
Kind regards