ECML/MML major power problems (09/08)

[sharpley]

According the the BBC News this morning, it was the National Grid that had to decide whether to take many households off the grid, or shut down the Power to Network Rail, they took the decision to cut the power to the rail network, they were or are syaing this decision will be looked at in the inquiry.

National Grid run the transmission network, which connects power generation to regional distribution networks (like Western Power, UK Power Networks and a few NR connections etc...) via bulk supply points. Its these bulk supply points that would have automatically switched off due to the drop in frequency, thus anything on the distribution network side (homes, hospitals, rail) would have been cut off. I guess any inquiry would look into which supply points were cut

 

[westcoaster]

Could the proximity of Little Barford power station to the MML & ECML have any bearing on the issue.

 

[AM9]

It's accepted practice that at times of shortage, energy suppliers will ration supply to industrial customers in preference to cutting off domestic households. It's down to those industrial customers to have equipment that doesn't take several hours to reboot.

Actually it's a bit more complicated than that. Large industrial users can volunteer to be near the top of the load-shedding list in exchange for a lower cost tariff. Industries where long slow processes with lots of inertia in their equipment can trade on that by tolerating short interruptions. (most load-shedding is quite short lived). Another aspect of power-bargaining is for voluntary self-restraint at peak times.

 

[AM9]

Could the proximity of Little Barford power station to the MML & ECML have any bearing on the issue.

Probably not as much as it might seem. I believe that Little Barford was the first loss and didn't in itself require any shedding. When automatic power-shedding does take place, it can produce islands of disconnection that might include a major NR grid connection, but not necessarily near the failed source,
Similarly, the Hornsea wind turbine farm has a DC-ac conversion at North Killinghome which is near Immingham, some way from the ECML and a very long distance from the most northerly part of the MML OLE, (Bedford).

 

[sharpley]

Could the proximity of Little Barford power station to the MML & ECML have any bearing on the issue.

No. Little Barford connects to the National Grid, not the local area distribution network. Thats why there were power cuts from Newcastle Airport down to London. NG couldn't maintain a 50Hz frequency so had to cut off regions to reduce demand and stabilise frequency. This was an automatic disconnect so I guess there will be an enquiry into which regions got cut off.

Basically : national grid connects power stations to local / regional distribution networks. As long as there is enough power generation on the national grid to meet demand on the distribution networks there shouldn't be a problem. power stations are decommissioned or switched off for maintenance all the time. doesn't mean the area where they are located loses power. Was an unfortunate circumstance that NG lost 2 major generators within minutes.

 

[edwin_m]

Aren't there discounts for industrial customers who agree in advance that they can be cut off first in this sort of situation?

 

[apk55]

I would also expect the grid to to drop the voltage the minimum possible before disconnecting loads. A 1% voltage drop will drop power by 2% for a resistive load. And this would also protect transformer cores from going into saturation something which has been discussed earlier. Unfortunately a lot of modern loads are constant power using electronic regulators - and this includes many things from trains to LED lighting

 

[youngpete]

Voltage reduction is only used when the load is building and it becomes obvious that there is insufficient generation to meet the predicted demand. In the case of the event recently action had to be taken immediately and thus the nationwide Low Frequency Protection kicked in and immediately shed predesignated bulk supply points or part thereof. Certainly in some areas National Rail supplies are protected and not shed.

 

[hwl]

Voltage reduction is only used when the load is building and it becomes obvious that there is insufficient generation to meet the predicted demand. In the case of the event recently action had to be taken immediately and thus the nationwide Low Frequency Protection kicked in and immediately shed predesignated bulk supply points or part thereof. Certainly in some areas National Rail supplies are protected and not shed.

Some NR supplies share bulk supply points and some have exclusive ones (e.g. 3rd rail supplies in south London with the 33kV feeds fed from their own 275/400kV transformers) so being able to segregate NR isn't always possible especially if fed from DNO network far away from NG infrastructure.

 

[Nicholas Lewis]

Excerpt below allegedly from the NG report into last Fridays grid incident as reported on a renewable energy website

"The report is also expected to show that eight rail-signalling networks
lost power supplies across the country, leading to much of the chaos on
the railways in England and Wales.

Power to overhead lines on train tracks were, however, largely
maintained, but the issue was exacerbated by a number of trains that
shut down automatically and could not be restarted by their drivers.
Govia Thameslink, which had as many as 60 trains affected, said it had
to call out engineers to restart half of those that failed.

Steve White, chief operating officer at the train group, said that their
own preliminary investigation had established a drop in electrical
frequency on the overhead power lines, which triggered safety mechanisms
on the shut down trains"

 

[initiation]

Yep, you can read the preliminary report here...
https://www.ofgem.gov.uk/system/files/docs/2019/08/incident_report_lfdd_-_summary_-_final.pdf

 

[KingJ]

Yep, you can read the preliminary report here...
https://www.ofgem.gov.uk/system/files/docs/2019/08/incident_report_lfdd_-_summary_-_final.pdf

Interesting read! For anyone looking for the specific impact on rail;

Rail

Network Rail and the DNOs have confirmed that no track supplies were lost due to the DNO’s LFDD protection operation. However, there were significant impacts on the rail network during the event as noted below:

1. While the built-in resilience of Network Rail’s electrical power infrastructure meant traction power was maintained to the vast majority of the railway throughout the incident, there were frequency-related trips which occurred at two DC traction locations which Network Rail are investigating.
2. Eight signal power supplies in principally rural locations suffered minor outages with minimal passenger impact. Network Rail are reviewing resilience at these locations;
3. Class 700 and 717 trains shut down north of Farringdon and Kings Cross stations due to their internal protection systems being triggered. The Network Rail overhead line power supply operated continually. The shutdown of these trains had a knock-on impact by delaying all other trains behind them requiring the temporarily closure of London St Pancras and Kings Cross stations which led to Friday rush hour overcrowding.

Govia Thameslink Railway (GTR) have advised that their Class 700 and Class 717 trains that were operating on AC power were affected by the frequency deviation below 49Hz. They are investigating with the manufacturer the reason for the trains stopping when the frequency dropped below 49Hz. Of the approximately 60 Class 700 and Class 717 trains operating on overhead power supply, half were restarted by the driver and the remainder required a technician for reset. All other GTR classes of train were unaffected. GTR will provide further information, relevant to the technical report, once their investigation has concluded.

The impact to the rail network was that thousands of passengers had their journeys delayed with 371 trains cancelled, 220 part cancelled, and 873 trains delayed. London St Pancras and King’s Cross stations had to close for several hours due to overcrowding and London Euston went exit only for a period of time.

London Underground have confirmed there were impacts on the London Underground Victoria Line, which was suspended as a result of the event and service was restored at 17.35.

 

[hwl]

Interesting read! For anyone looking for the specific impact on rail;

Worth nothing issues with DC rectification for 3rd rail in some places due to low frequency... Not just an on train issue.

 

[AM9]

It looks like National Grid recognise the new environment under which electricity is generated and distributed. Given the high cost of providing inertia on a national scale that used to be 'free' with less green generation methods, this was maybe a wake-up call, not only to the government (via Ofgem) but to users that have equipment and working practices that need to operate with a less frequency stable supply.
For the supply side, there would seem to be a greater margin of fallback generation required that would seem to indicate greater investment in the system, (government to note).
For the use side, and specifically in this thread, Network Rail/train manufacturers, must ensure that the ever increasing optimisation of designs to reduce power and weight comes with adequate protection that must work, maybe with some form of recording of local operating conditions that are causing shutdowns, that can communicate back to support areas, and by some means authorise exceptional local recovery actions in such a way that those on the ground (i.e. drivers) can safely undertake them. This is not specifically a Siemens issue as all manufacturers are in the same battle to square the circle on conflicting requirements of performance, economy, costs and of course relaibility/availability.

 

[HSTEd]

I'm not sure I like the margins this report suggests are adequate.

Single power block tripping out kinda ignores nasty common mode failures that can easily take out multiple units on the same site.

It looks like National Grid recognise the new environment under which electricity is generated and distributed. Given the high cost of providing inertia on a national scale that used to be 'free' with less green generation methods, this was maybe a wake-up call, not only to the government (via Ofgem) but to users that have equipment and working practices that need to operate with a less frequency stable supply.

The cost of providing frequency stability should have been priced into these new generation and distribution schemes.
In essence, wind turbines and solar panels have been freeloading on synchronous generators, and now rather than paying for this externality the public are just expected to absorb inferior quality of service.

These generators should be required to provide grid inertia comparable to that produced by synchronous generators or they should be required to pay for it to be provided by the Grid Operator.
We shouldn't accept the degradation of the grid system so some wind-pushers can lie about how their grid integration costs are tiny (since it turns out they haven't been paying them at all).

And there is nothing "less green" about synchronous generators.
About the only reason the grid didn't fall over completely was because of the huge turbine trains provided by the nuclear fleet.

 

[edwin_m]

Could and should the generators who use solid state inverters have more responsibility for maintaining frequency? I'm thinking maybe the inverters could run ever so slightly out of phase with the grid when the frequency was off 50Hz, in an attempt to drag it back.

 

[AM9]

Could and should the generators who use solid state inverters have more responsibility for maintaining frequency? I'm thinking maybe the inverters could run ever so slightly out of phase with the grid when the frequency was off 50Hz, in an attempt to drag it back.

I don't think it quite works like that. A small number couldn't make much difference anyway apart from putting distortion onto the waveform which would create EMC and power factor issues everywhere. To do that otherwise would require the majority of generators all working in perfect sync, and they aren't designed to be dragged around in a phase lock system for thatt would reduce their efficiency.

 

[AM9]

I'm not sure I like the margins this report suggests are adequate.

Single power block tripping out kinda ignores nasty common mode failures that can easily take out multiple units on the same site.



The cost of providing frequency stability should have been priced into these new generation and distribution schemes.
In essence, wind turbines and solar panels have been freeloading on synchronous generators, and now rather than paying for this externality the public are just expected to absorb inferior quality of service.

These generators should be required to provide grid inertia comparable to that produced by synchronous generators or they should be required to pay for it to be provided by the Grid Operator.
We shouldn't accept the degradation of the grid system so some wind-pushers can lie about how their grid integration costs are tiny (since it turns out they haven't been paying them at all).

And there is nothing "less green" about synchronous generators.
About the only reason the grid didn't fall over completely was because of the huge turbine trains provided by the nuclear fleet.

There's nothing particularly green about coal*, oil and gas as CO2 generators that is green. Once gas domestic and industrial heating and hydrocarbon burning traction is largely removed from the energy train, CO2 will at last be under control.
*coal power stations will be completely green soon as they will all be gone. Wind/solar/tidal and the little bit of nuclear that this country manages to get, are the only long-term methods that will bring CO2 levels back under control. Casting aspertions about lying 'wind pushers' doesn't change that.

 

[HSTEd]

There's nothing particularly green about coal*, oil and gas as CO2 generators that is green. Once gas domestic and industrial heating and hydrocarbon burning traction is largely removed from the energy train, CO2 will at last be under control.
*coal power stations will be completely green soon as they will all be gone. Wind/solar/tidal and the little bit of nuclear that this country manages to get, are the only long-term methods that will bring CO2 levels back under control. Casting aspertions about lying 'wind pushers' doesn't change that.

Equating synchronous generators with carbon emission is disengenous.
Wind sold itself as being better than nuclear by virtue of being cheaper, and yet it turns out that they have been loading externalities onto the grid, and now that those externalities have been made apparent -t hey should pay rather than demanding the public pick up the tab.

 

[edwin_m]

I don't think it quite works like that. A small number couldn't make much difference anyway apart from putting distortion onto the waveform which would create EMC and power factor issues everywhere. To do that otherwise would require the majority of generators all working in perfect sync, and they aren't designed to be dragged around in a phase lock system for thatt would reduce their efficiency.

The existing generators whether rotary or solid-state inverters must run in sync with the grid to prevent problems of this type. I'm not sure how frequency is maintained historically or currently (no pun intended) but it must surely involve getting some generators to push a bit more power when the frequency drops, and that would seem to imply still running in sync but slightly ahead of the phase of the network as a whole.

There's a range of stuff on the National Grid website but it's a bit technical!

 

[initiation]

it must surely involve getting some generators to push a bit more power when the frequency drops,

Traditionally, the large inertia of many steam generators (tonnes of metal rotating very quickly) would provide an immidiate short term response to a drop in frequency by slowing down slightly.

Conventionally, wind and solar don't operate in this way. However you can create 'synthetic' or 'digital' inertia using the power electronics in these systems, or by utilising batteries (which can provide a response faster and of greater magnitude than conventional inertia can). This hasn't been done on a large scale yet because there is no recruitment for such, but it could be.

https://spectrum.ieee.org/energywise/energy/renewables/can-synthetic-inertia-stabilize-power-grids

http://s2.q4cdn.com/601666628/files/doc_presentations/2017/Everoze-Batteries-Beyond-the-Spin.pdf


Hopefully, the TOCs and manufacturers will learn from this event and get kit operating as it should be, even under unusual (but not unforeseen) circumstances.

 

[AM9]

Traditionally, the large inertia of many steam generators (tonnes of metal rotating very quickly) would provide an immidiate short term response to a drop in frequency by slowing down slightly.

Conventionally, wind and solar don't operate in this way. However you can create 'synthetic' or 'digital' inertia using the power electronics in these systems, or by utilising batteries (which can provide a response faster and of greater magnitude than conventional inertia can). This hasn't been done on a large scale yet because there is no recruitment for such, but it could be.

https://spectrum.ieee.org/energywise/energy/renewables/can-synthetic-inertia-stabilize-power-grids

http://s2.q4cdn.com/601666628/files/doc_presentations/2017/Everoze-Batteries-Beyond-the-Spin.pdf


Hopefully, the TOCs and manufacturers will learn from this event and get kit operating as it should be, even under unusual (but not unforeseen) circumstances.

Not necessarily easy as (presumably), the class 700's protection mechanisms acted to prevent transformer damage which could have been anything from a burnt out winding to worst case, a full-scale fire. Transformers in EMUs have an unfortunate history in the uk and remember, those on the Desiro City's were said to have been designed for minimum weight and size yet still deliver 2.5MW! That's enough to power a sizeable village which would have a static oil-cooled transformer in a fenced-off enclosure, - about 1/4 of the vloume of a 20m coach. As far as restarting goes, for the same reason, it would be essential to:

a) check that an out of spec condition had been removed
and
b) establish that the device was safe to turn on in its own right.​

That might be the reason why the traction management system would't let the drivers just reboot and get underway.

 

[edwin_m]

Not necessarily easy as (presumably), the class 700's protection mechanisms acted to prevent transformer damage which could have been anything from a burnt out winding to worst case, a full-scale fire. Transformers in EMUs have an unfortunate history in the uk and remember, those on the Desiro City's were said to have been designed for minimum weight and size yet still deliver 2.5MW! That's enough to power a sizeable village which would have a static oil-cooled in a fenced-off enclosure, - about 1/4 of the vloume of a 20m coach. transformer As far as restarting goes, for the same reason, it would be essential to:

a) check that an out of spec condition had been removed
and
b) establish that the device was safe to turn on in its own right.​

That might be the reason why the traction management system would't let the drivers just reboot and get underway.

If the transformer is physically incompatible with anything below 49Hz then there's probably no scope to change it, and little need to considering this has only happened once in the 2+ years these units have been in service (and presumably there's a record somewhere of previous occurrences to establish a probability per year). Unless these events are going to become more frequent (and NG is presumably tasked to prevent that) a mass Desiro shutdown every few years is probably tolerable as long as service can quickly be restored.

One factor may be whether this design can monitor the supply frequency when its VCB is open. If not, then the transformer has to be connected to a possibly suspect supply to establish whether it is still suspect, and there may be concern about causing damage if this is done more than a couple of times. There's also the issue of whether it's just situation everyone assumed would never happen based on interpreting an ambiguous spec, or a genuine possibility of damage if too many re-boots are attempted when the supply is out of range.

 

[Class 170101]

It looks like National Grid recognise the new environment under which electricity is generated and distributed. Given the high cost of providing inertia on a national scale that used to be 'free' with less green generation methods, this was maybe a wake-up call, not only to the government (via Ofgem) but to users that have equipment and working practices that need to operate with a less frequency stable supply.

To a point doesn't wind provide inertia? The only way I can see to provide more secure 'inertia' is through pumped storage, batteries and possibly tidal.

In terms of stability of supply. It was August 9th - a summer's day and yet power was lost from two generators of it. Now imagine the same day at the beginning of February where more power is probably needed....

 

[hwl]

To a point doesn't wind provide inertia? The only way I can see to provide more secure 'inertia' is through pumped storage, batteries and possibly tidal.

In terms of stability of supply. It was August 9th - a summer's day and yet power was lost from two generators of it. Now imagine the same day at the beginning of February where more power is probably needed....

Wind all goes via an inverter and DC (especially with under water cables /skin effect). (And is geared.)
Nothing provides inertia quite like a steam turbine rotor @ 3,000rpm (and when it fails you start looking in fields several miles away...)

 

[edwin_m]

In terms of stability of supply. It was August 9th - a summer's day and yet power was lost from two generators of it. Now imagine the same day at the beginning of February where more power is probably needed....

If the total load was greater the loss of two generators would be less as a proportion of that total. Lightning is now being suggested as contributory, which would be less likely in winter.

 

[Nicholas Lewis]

If the transformer is physically incompatible with anything below 49Hz then there's probably no scope to change it, and little need to considering this has only happened once in the 2+ years these units have been in service (and presumably there's a record somewhere of previous occurrences to establish a probability per year). Unless these events are going to become more frequent (and NG is presumably tasked to prevent that) a mass Desiro shutdown every few years is probably tolerable as long as service can quickly be restored.

One factor may be whether this design can monitor the supply frequency when its VCB is open. If not, then the transformer has to be connected to a possibly suspect supply to establish whether it is still suspect, and there may be concern about causing damage if this is done more than a couple of times. There's also the issue of whether it's just situation everyone assumed would never happen based on interpreting an ambiguous spec, or a genuine possibility of damage if too many re-boots are attempted when the supply is out of range.

Whats really needed here is for ORR and DoT to hold GTRs feet to the fire like OFGEM has done to NG backed up by DoBEIS such that there is transparency to what happened. However, its convenient for GTR aided and abetted by DoT to hide behind the power cut so will we ever really find out what the root cause was?

 

[AM9]

Whats really needed here is for ORR and DoT to hold GTRs feet to the fire like OFGEM has done to NG backed up by DoBEIS such that there is transparency to what happened. However, its convenient for GTR aided and abetted by DoT to hide behind the power cut so will we ever really find out what the root cause was?

To be fair, GTR don't appear to be guilty of anything on the 9th August, except maybe for a lack of passenger information, but even that would be dependent on NR and their source, Natonal Grid.

 

[Nicholas Lewis]

To be fair, GTR don't appear to be guilty of anything on the 9th August, except maybe for a lack of passenger information, but even that would be dependent on NR and their source, Natonal Grid.

Indirectly maybe not as its Siemens product but the fact remains that the supply disturbance caused the cancellation of 371 trains, 220 part cancelled and 873 trains delayed (several LNERs in excess of 4hrs) making this is very significant event for our industry. GTR are the customer facing organisation and thus should be communicating what the cause was and what action needs to be taken to provide both future resilience and the ability of the driver to be able to deal with the situation.

 

[AM9]

Indirectly maybe not as its Siemens product but the fact remains that the supply disturbance caused the cancellation of 371 trains, 220 part cancelled and 873 trains delayed (several LNERs in excess of 4hrs) making this is very significant event for our industry. GTR are the customer facing organisation and thus should be communicating what the cause was and what action needs to be taken to provide both future resilience and the ability of the driver to be able to deal with the situation.

No not necessarily. The trains were specified to run on UK supply specifications at the time of their ordering, (by the DfT, not GTR). In the document that I referenced in post #5 of this thread:
https://www.railforums.co.uk/thread...d-sufficiently-resilient.188283/#post-4146732 it seems that Ofgem have allowed National Grid an eight-times specification relaxation on maximum change of frequency, (from 0.125 Hz/s to 1.0 Hz/s) which since that publication was in 2014, it is possible that this month's problem was the first time that the faster rate of change had happened. If that is so then were Siemens told before the trains were delivered (or even since). The DfT would have primary responsibility for that as it would be a technical change to the contract. I suspect that if this change was a factor in the whole event, the Ofgem and DfT enquiries will untimately suggest a way ahead.
If Siemens had no knoweledge of the change, then it would be reasonable for them to regard such a rate of change as a) exceptional and more importantly b) a situation that not only requires immediate disconnection from the supply, but also if repeated (on attempted reboots), indicative of an unknown issue and requiring investigation before allowing a reconnection. Bear in mind that were a transformer to be subjected to out of frequency range power under load, we could be discussing why so many class 700 trains had their transformer catch fire.
So, much as you seem to want to blame GTR for all of this, their only real failure was in the dissemination of information to stranded passengers et al. That of course is something that in general, every responsible organisation had problems with on the day. As for trying blame somebody for delaying trains along the whole of the ECML, forget it. It doesn't take many unforseen breakdowns to to bring any busy mainline to its knees. These things happen all over the world occasionally, - get over it, - and nobody was killed or seriously injured as far as we know, (except a few individuals' pride maybe).