Electrical Supply Problem(s) 02 May

[JustSomeBro]

It appears that there are some major issues with Elizabeth Line suspended throughout and major delays on the Greater Anglia network.

Due to electrical supply problems:

Train services running across the whole Greater Anglia network may be cancelled, delayed or revised. Disruption is expected until further notice.

Owing to electrical supply problem across the Greater Anglia Network to the electrical overhead lines train services across multiple routes are subject to delays, alterations and cancellations.

Network Rail are currently investigating the issue.

I'm guessing it’s connected?

 

[JetTheWolf]

National Rail claiming that there was (or is) a "nationwide power fluctuation with National Grid". Something must've tripped somewhere major

 

[JustSomeBro]

I’m on the GWML and the relief lines look like a carpark for all the Elizabeth Line trains. Something gone wrong big time somewhere…

 

[GraphitProjekt]

Guessing it’s connected?

Yeah, the traction power for Stratford to Gidea Park/Upminster is the same supply as the entire Crossrail Core, the Pudding Mill Lane 400kV site.

There are backups though, the XR Core can be fed from the GWML supply at Kensal Rise and I think GE might be able revert to it's 132kV equipment that it used to use before 2018ish (Bow Junction and Crowlands) at least with a reduced service.

But I'd suspect this kind of issue is where the power cut is relatively short, but staff have to go and assess if there's any damage or if it's fine to go back to the normal arrangement or institute emergency feeding arrangements.

Another interesting point is that apparently the West Coast Mainline is affected in some way by the same incident according to National Rail
https://www.nationalrail.co.uk/service-disruptions/power-outage-20260502/

There was a power outage earlier today which has caused disruption to some routes across London and the Midlands. Services are now on the move, but may still be severely impacted. The affected train operators include:

• Avanti West Coast
• c2c
• Elizabeth line
• Greater Anglia / Stansted Express
• London Northwestern Railway
• London Overground
• West Midlands Railway

 

[GraphitProjekt]

Looks like the Weaver Line is also shut down according to TfL status page. I assume because of the wider restriction on Liverpool Street rather than with the WAML supplies. This is because apparently the Mildmay Line is okay throughout

 

[Vexed]

Apparently there was a quick (well, quickish, not as quick as the Thameslink meltdown in August 2019) reduction in frequency down to 49.6 Hz just after 4pm. I wonder if this is related? The timing lines up.

https://x.com/i/status/2050602047905439955

UK #nationalgrid suffered a bit of an incident around 4pm BST, dipping below 49.6 Hz, but not breaching the statutory minimum of 49.5 Hz.

 

[Peter Mugridge]

Given there's a few storms around this afternoon and evening, could part of the grid have taken a lightning strike leading to protection trips around it tripping out?

Have heard the odd rumble or two here in Epsom, although I know we're on the opposite side of London from the affected area!

 

[172007]

West Midlands also affected according to Network Rail.

 

[ANDREW_D_WEBB]

Apparently there was a quick (well, quickish, not as quick as the Thameslink meltdown in August 2019) reduction in frequency down to 49.6 Hz just after 4pm. I wonder if this is related? The timing lines up.

https://x.com/i/status/2050602047905439955

Almost certainly. I was trying to get on the Elizabeth line at 16:10 when it all got suspended.

 

[172007]

Almost certainly. I was trying to get on the Elizabeth line at 16:10 when it all got suspended.

Was just looking at Gridwatch Templar and the frequency as it was my hunch, does not show the frequency as a graph through tje day. Only what it is now.

 

[Tilting007]

On WCML affected 730s only.

 

[Ze Random One]

Opentraintimes maps suggests that a restart of the Elizabeth line is being attempted. Not much in the core, but the Shenfield line now has quite a few trains about on the slows

 

[GraphitProjekt]

Given there's a few storms around this afternoon and evening, could part of the grid have taken a lightning strike leading to protection trips around it tripping out?

Have heard the odd rumble or two here in Epsom, although I know we're on the opposite side of London from the affected area!

Could be, but the only piece of grid infrastructure that seems to have fully failed was Pudding Mill Lane which feeds all of XR core and all lines out of Liverpool Street. Pudding Mill Lane is fed from cables of course.
The problems also affected WCML but it's hard to figure out what that was or how it's related.

The Pudding Mill Lane failure seems to be somewhat localised because Kensal Green (for GWML) seems to have been fine throughout. The feeds for other lines like C2C (West Ham, Barking), WAML/ECML (Tottenham) are connected at the 132kV level and so are a little more insulated from whatever issue affected the transmission network. We also know that Bow Junction feeder was still operational as the Mildmay Line was running, that one is downstream of Hackney substation.
The WCML issue is illusive because the London are is fed downstream from Willesden, but this also feeds the North London Line. Elstree substation has supplies for WCML and MML (Bushey and Borehamwood respectively) and no issues reported on MML. Leighton Buzzard and Patford Bridge are dedicated National Grid feeds for WCML further north.

On WCML affected 730s only.

This makes a lot of sense. If it only affected one kind of train then I suspect the 730s responded differently to the nationwide frequency drop compared to older trains. Similar to the Thameslink meltdown in 2018. Perhaps we can say that none of the WCML supplies actually failed.

 

[Vexed]

Frequency dropping to 49.6 Hz is a much smaller drop than the August 2019 class 700 issues, which was 48.8 Hz. While usually kept within 0.2 Hz of the target 50 Hz, NESO (National Energy System Operator; previously known as National Grid) is only required keep it within 0.5 Hz, which they did do (assuming no momentary drops not covered in the graph I posted in #7). Iirc, the 700s were supposed to handle down to 48.5 Hz with a simple driver reset, no fitter attention required, so if the 730s can't handle a much smaller drop we really didn't learn did we...

I will caveat that the frequency drop might have been accompanied by some other disturbances that did cause either the feeders to trip or the trains themselves, such as harmonic distortion.

 

[saismee]

It was very much just a 20-minute blip but it required a reboot of all trains in the affected area.

I will caveat that the frequency drop might have been accompanied by some other disturbances that did cause either the feeders to trip or the trains themselves, such as harmonic distortion.

All pans dropped automatically.

 

[iamraymond]

It was very much just a 20-minute blip but it required a reboot of all trains in the affected area.

All pans dropped automatically.

What happens here, do they coast to a stop? Or everything turns red?

 

[saismee]

What happens here, do they coast to a stop? Or everything turns red?

Someone better qualified can give specifics (I've stepped foot in the cab just about twice), but I believe ADD (automatic drop device) activations cause the emergency brake to apply. I imagine signal, if fed from the same supply, would appear blank.

 

[GraphitProjekt]

I wonder if Elizabeth Line's class 345s were also affected by the frequency event, or maybe some other rail systems on the XR core. Because if you look at the RTT data just for GA in/out of Liverpool Street, you can tell things started to go awry around 1620 (some 10 minutes after the frequency came back up, perhaps the Grid's restoration process caused something to trip at Pudding Mill Lane specifically). However, the number of trains actually cancelled is not especially high and the delays are in the region of 15-20 minutes for most trains and it's pretty much cleared up by now, although there has been a absolutely tidal wave of platform alterations that are continuing throughout the night.

Whereas looking at Crossrail at say Tottenham Court Road it looks like a total shutdown from around 1605 whereas GA trains were still departing on time until 1620. I wonder if this is also a success story for GA's recovery tactics, because on paper it doesn't look excessively bad as it does on XR. I noticed that some Southend trains started/ended early at Shenfield which looks like a standard contingency plan put into use.

It looks to me like XR was affected by the low frequency and then additionally the loss of Pudding Mill Lane, whereas only the latter was a problem for GA class 720s and 745s.

but I believe ADD (automatic drop device) activations cause the emergency brake to apply. I imagine signal, (if fed from the same supply, would appear blank.

I haven't seen any evidence that signalling was affected by the frequency drop - could be wrong though. In my opinion, I don't think any power supplies were actually lost in the West Midlands or WCML rather than 730s reacted to the frequency drop: power was still available but just not to their very specific tastes.

 

[saismee]

I haven't seen any evidence that signalling was affected by the frequency drop - could be wrong though.

I doubt it would've been affected unless the substation (?) powering the equipment dropped out. I can't see anything internal to suggest that signals were affected, though they're not very high-priority if the trains themselves can't move either.

 

[GraphitProjekt]

I doubt it would've been affected unless the substation (?) powering the equipment dropped out. I can't see anything internal to suggest that signals were affected, though they're not very high-priority if the trains themselves can't move either.

I'd counter that a signalling power outage would be of even higher priority than loss of traction power because signallers and controllers can no longer see where trains are, even if electric ones have stopped (or in this case, only class 730s) plenty of diesel still moving.

I don't know a lot specifically about signalling power, but on electrified lines signalling supply points (SSPs) are directly connected to 25kV OLE circuits and step it down to either 400V or 650V. But signalling systems additionally connect to the local distribution grid (11kV) and interlocking rooms may have battery backups or even diesel generators.

But again, I don't personally believe there was any outage in the West Mids - just equipment reacting to the drop in frequency. It's possible though perhaps unlikely that some signalling equipment could react badly to this too - I wonder if this happened on the XR Core because their stuff seems to be often quite troublesome.

 

[saismee]

The XR core and most of the GEML are mostly electric-only, and any diesel train would likely be sandwiched between affected electrics. I would hope that anyone competent to drive a freight train would very quickly notice a blank signal and activate a REC (which may have already happened with multiple ADD activations around the area). Until the electrics can move the diesels can't either, so the signalling isn't super relevant.

signallers and controllers can no longer see where trains are

This is quite a fair point, especially with the risk of stranded or rowdy passengers (QPR loss comes to mind) detraining themselves. I also realised after writing the above that there is more than just the XR core and southern GEML directly affected, and that other nearby branches were likely affected.

 

[GraphitProjekt]

Until the electrics can move the diesels can't either, so the signalling isn't super relevant.

I know you crossed it out, but there really isn't any scenario where you can say loss of signalling isn't relevant.

If trains are pulling emergency brakes either because of the driver seeing a blank signal (which are considered red) or from ADD activations, the trains aren't going to stop in the last signalling block reported before the signaller lost the ability to detect trains. No signalling could also affect telecoms though I know even less about that.

But in general, I don't think there is a possible failure mode linking traction power and signalling to one cause since signalling always has local grid backups or occasionally generators. Power fluctuations for signalling will surely be mitigated by UPSs as well.

Also, while a single 25kV feeder station will be supplying up to like 40km in either direction, signalling power is taken at a much smaller level, so to have the equivalent number of signalling PSPs drop out sounds seriously unlucky - maybe even impossibly so.

 

[saismee]

No signalling could also affect telecoms though I know even less about that.

If the signalling equipment has backups I'm 100% certain that telecoms would have even more backups.

Overall though it wasn't too major of an incident for how large-scale it could've been. Imagine if it was the TL core instead of XR... we would likely still be feeling the delays and cancellations now.

 

[GraphitProjekt]

Looking at Tottenham Court Road, about 15 of the 33 train movements in the hour of 8pm were cancelled, throughout 5pm there were 40 movements and only about 3 trains weren't cancelled. Whereas on GA it was nowhere near that bad. At Liverpool Street High Level, there were 36 movements throughout 5pm of which 6 were cancelled and the rest ~15mins late. So I'd say that XR have faired pretty badly, they seem to be back in order by now though.

 

[iamraymond]

I haven't seen any evidence that signalling was affected by the frequency drop - could be wrong though. In my opinion, I don't think any power supplies were actually lost in the West Midlands or WCML rather than 730s reacted to the frequency drop: power was still available but just not to their very specific tastes.

27 minutes late, wrong type of electricity at Selly Oak - very Reggie Perrin

 

[rob.rjt]

As usual the Elizabeth Line contingency arrangements were awful on the GWML. Ticket acceptance was arranged on London buses only, which is no good for stations west of West Drayton, or GWR services which don't serve the smaller stations. As far as I can tell, GWR didn't add calls to the Didcot services to serve the smaller stations normally solely served by Elizabeth Line.

 

[TFN]

As far as I can tell, GWR didn't add calls to the Didcot services to serve the smaller stations normally solely served by Elizabeth Line.

GWR did have special stop orders on a few of their Didcots at Ealing Broadway, Southall, Hayes & Harlington, West Drayton, Langley and Iver.

 

[rob.rjt]

GWR did have special stop orders on a few of their Didcots at Ealing Broadway, Southall, Hayes & Harlington, West Drayton, Langley and Iver.

Interesting - other than Langley and Iver they would be covered by ticket acceptance on London buses. No covering service west of Slough though.

 

[Belperpete]

I know you crossed it out, but there really isn't any scenario where you can say loss of signalling isn't relevant. If trains are pulling emergency brakes either because of the driver seeing a blank signal (which are considered red) or from ADD activations, the trains aren't going to stop in the last signalling block reported before the signaller lost the ability to detect trains. No signalling could also affect telecoms though I know even less about that.

Actually, it is probably more the other way around - loss of telecoms would likely affect signalling. Signalling relies heavily on telecom links between control centres / signalling centres and the trackside equipment. As does electrical power switching, CCTV surveillance, and a host of other systems. All such control and telecom equipment nowadays is computer based, and the critical equipment at least will be fully surge and UPS protected.

The only telecom things that I can think of that would be affected by loss of signalling would be things driven by the train describer, which derives its stepping information from the signalling. Things like passenger information systems, cab radio, etc. Nowadays the train describer tends to be an inbuilt function of the control system, rather than a stand alone unit.

I can understand that it is impractical to provide equivalent protection to traction motor supplies, but surely the control equipment on rolling stock should be properly protected?

 

[a_c_skinner]

This hasn't been reported on national news I've seen. Was all this simply the dip in grid frequency and does anyone know what caused it? Why is such a modest (tiny?) drop in frequency such a deal breaker for modern trains?

This is going to be more of a problem as the huge inertia of the big synchronous turbine generator sets of fossil fuel stations and the current nuclear stations go off line. I think this lack of inertia in the system caused the Iberian black out of a few years ago and makes restarting a grid much harder.