Contrast this deliberate act to make interchange harder with the good hearted railway designers in the mid 20th century. Central line to BR at Stratford,Victoria line to northern line at Euston, Vic line to Pic line at Finsbury Park etc etc, all 20 second cross platform interchanges on the level & achieved at considerable engineering cost. Then look at the convoluted Jubilee line interchanges, mostly designed to allow ticket checks on travellers between the different lines. Then look at the latest incarnations. The seemingly mile long interchanges under Kings Cross. The aforementioned London Bridge main line to tube extended hikes & the very recent pushing towards the country of the Waterloo international platforms so more retail can soon be jammed in. Oh and also the wonderfully simple interchange at Paddington 'Hammersmith & City' island platform to the main Line platforms via a footbridge now another external hike.. I often wonder if railway designers, who used to care about people as evidenced by their thoughtful designs now just regard punters as a nuisance.
Alternatively, perhaps the designers don't deliberately try to make it difficult for passengers. Perhaps they design things that are intended to work with the numbers of people expected to use them.
Take London Bridge as it is the topic. To get the necessary number of platforms, and a good width on each of them meant pushing the through platforms a little to the east. Going west was not practical because of the lines converging at Borough Market Junction. Unless you wanted to demolish Southwark Cathedral and build a new bridge across the Thames that is.
Given that the platforms had to move east, the concourse has to go with it. Passenger numbers required 2 banks of escalators on the through platforms, and to spread people out on the platforms the escalators need to be separated at the top, ie being closer together at the bottom. This combined with platform width requirements fixes the western escalator head, and thus the westernmost point of the concourse. If they could have gone further west they would.
Some long interchanges at busy stations are a fact of life. It is physically impossible to get all platforms of a multi line interchange close to each other. The various tunnels have to fit around each other, the necessary alignments of the running tunnel approaches, passageways, building foundations, other utility tunnels (of which there are a great many), and, not least, geology. Kings Cross / St Pancras being the best example, with five separate lines serving 8 separate station tunnels underground, a very deep basement next door at the British Library, another one up at the Francis Crick Institute, 2 main line stations 'upstairs' with 24 platforms between them, various utility tunnels and a river all in close proximity.
Finally, Waterloo hasn't been pushed out for more retail, it has been done to create more concourse space at the UKs busiest station, and also to maximise capacity on the track layout at the throat.