I travel between the two most weekends
Apologies, I read that you were looking to travel to/from the Olympics, but are actually looking for an alternative
I would suggest you look at the Olympics schedule, I would expect the second half to be busier as people attend the Athletics events
There will be a queuing system in effect for passengers between St Pancras and Stratford, with Games Travelcards, so you will probably want to avoid this
Therefore it might be more appropriate to travel (LU) Paddington to Notting Hill Gate, the via Liverpool Street to Stratford