Surely any public transport system that has to 'design in' space for many hundreds of passengers to just stand around for lengthy periods on a regular basis has fundamentally failed. (I am obviously not talking about occasional events like Glastonbury crowds at Castle Cary.)
The frequency and reliability of services both in and out of an interchange should allow most users to make relatively continuous progress.
There will always be a proportion of passengers who will be around for longer and these need to be both catered for and 'managed' to ensure that there is no continuous 20-deep scrum around the barriers.
General terminal planning at airports (separation of arrivals and departures) and major stations has developed to use multi-level formats with facilities designed to encourage at least a significant share of people to 'stay out of the way until the right juncture'. The availability and clarity of information is a major part of this art.
In general terms I think that St Pancras achieves this very well. As a regular user I will often mooch across to King's Cross for a bit of variety of coffee houses, bars and shops too. With Real Time Trains on my phone I neither stand around aimlessly nor miss my train.
I understand why airports and now train stations are doing this, to maximise retail opportunities. It's why airport security has been speeded up so much, and not because airports wanted to help us out!
People don't rush to the plane the same as they did when low cost airlines had a free for all on seating. On trains, there's still a fear of losing your seat, luggage space or even having your reserved seat taken. Whatever you do, you're unlikely to change that behaviour and designers need to take into account human nature, and study passenger behaviour.
I was there before it opened to do the user testing for Eurostar, and our feedback would have contributed to a few changes - including better signage. Initially St Pancras, like Heathrow Terminal 5, had minimal signs and people got massively confused. BAA at the time even produced little maps to hand out, proving they had failed spectacularly.
More open space isn't so much to encourage more people to linger around, but more to cope with the numbers already there during disruption OR when TL trains are terminating upstairs too. Last Sunday was a mess, although I went through when they hadn't closed the gates and waited on the platform because there was no need to clean a 700 on arrival or put out reservation tickets.
You are right that many people are thankfully clued up, know where they're going and when. They have travelled before so aren't scared they'll miss their train/seat etc. These people aren't the problem as they're not up there now and wouldn't be then. It's just that things are too cramped today.