London is a unique case simply because its size and the fact it already had a population of 5 million by the time the car was invented means that driving a significant way across it is purgatory. The M25 may have eased road travel from one side to another, but that has filled up rather quickly.
Another key factor is that London has for a long time had a central core with a huge amount of desirable employment / attractions that people want to get to but don't want to live next door to. Consequently London hasn't really had any choice but to invest in mass transit if people want to get anywhere reasonably quickly. The fact that MPs need to get to Westminster and civil servants to Whitehall probably doesn't hurt either.
The expansion of office jobs in other cities is rather more recent. A lot of towns around Greater Manchester and the industrial bits of Yorkshire had until around the 1980s plenty of local industrial employment. Congestion into large cities probably wasn't as bad then as there wouldn't have been the same number of people needing to get there for work (admittedly I'm not from that part of the world so please do correct me if I'm wrong). As that has fallen away and not been replaced by much other employment in those towns, the need to provide decent links into Manchester, Leeds, Sheffield etc. for employment opportunities has grown, presumably with the resultant congestion.
There is surely an element of 'build it and they will come' with suburban and interregional lines though. The Devon and EMR examples given above are a good example of this, as is the Cross-City line in Birmingham - what was a sporadic elderly DMU operated service in the late 70s is a frequent electric service. The Severn Beach line in Bristol has seen a sizeable increase in patronage by running more frequent services more reliably (employing onboard ticket sellers / examiners probably also helps). Providing reliable commuter services where passengers can be confident of getting onboard, and reliable interregional services where passengers can be confident of getting a seat is always helpful. TFW and GWR's habit of putting a 2 coach 197 / 165 or 158 respectively on Cardiff - Manchester and Cardiff - Portsmouth services can't help confidence in the service. Granted it's usually that or cancelling the train, but having a longer train consistently turning up is always useful.
As for XC, there are still frequently 4 coach trains with 2.5 standard class coaches running Plymouth - Exeter - Bristol - Birmingham - Derby - Sheffield - Leeds - York - Newcastle - Edinburgh, and also Bournemouth - Southampton - Reading - Oxford - Coventry - Birmingham - Wolverhampton - Stoke - Manchester. Call me old-fashioned, but I can't help feeling that there should be enough people on those routes to fill that several times over if they could be reasonably sure of getting a seat. There must be huge suppressed demand on the XC axis (and yes I know there's at least one thread here about XC overcrowding every month).
Longer trains are an easy fix as long as they fit into platforms. It's frequently said that pretty much all of XC's problems would be fixed with longer trains and it's hard not to believe that.
How many journeys in London and the S.E are run with 40 plus year old two car units?
It's really just trains to and from Central London where this applies. GWR run 2 coach 30 year old diesel 165s on Newbury - Bedwyn shuttles, Reading - Basingstoke and the Thames Valley branches. Reading - Redhill trains are run with 3 coach class 165s but are frequently full and standing for a large part of the journey. Many of the Ashford - Hastings services are run with 2 coach 25 year old diesel 171s. As for the Marston Vale... There aren't many services in the SE which don't go into London, but those that avoid it aren't really better than services anywhere else.