Wrong-side signalling failure at Wingfield

[Surreytraveller]

Very similar in appearance yes. 'The railway', and all involved here have had a very lucky escape in my opinion.

The railway has to be lucky all the time. Fate only has to be lucky once

We are well overdue a big one

 

[Tomnick]

I d

I do . Read my posts. I know the driver very well and was travelling with. Received a contact signaller message and decided to call signaller as running on greens and slowing down for the drop to 100 before the drop to eighty . Signaller was hoping for 1F01 to stop at next signal which wasn't possible and ironically might have been possible if requested or called , rather than waiting for the driver to call. They had cautioned the previous stopping service but didn't for the next non stop fast train. They also knew it might be something bad as a network rail employee that boarded 1F01 after finishing a night at Derby told us that something bad was going on , but not exactly what. So the signaller was concerned enough to 1. Caution a previous train . 2 . Try and get 1F01 to stop at the next signal , but not enough to send anything other than a contact signaller message. Also the two trains must have been in the section and not just the incident with the first train as it's a mere three or four minutes from Belper to Wingfield in a fast IC.
Just thought I would add that the driver didn't realise the exact problem until back in London and as he was stopped and no incident with that train had nothing to report. Very upset though , so probably no reason to be interviewed.

Thanks. The important thing here though is that there's no Rule Book requirement for 1F02 to be advised at all, nor - in those first few minutes of the incident - any realisation that there was anything amiss with the signalling system that'd justify advising 1F02. It was only the initial assumption that it was a Cat A SPAD (in old money) combined with the local knowledge of the signaller on the adjacent workstation that led to the Derby signaller doing it. We should be eternally glad that they did, because otherwise we'd almost certainly have been looking at a collision, but I don't know how you could ever make it an absolute requirement to do anything like that.

 

[68000]

I was an SMTH tester over 20 years ago and almost all signalling technicians and testers use head torches for location case lighting for work of this nature. I must say I am surprised that certified signal technicians or testers did not know the difference between internal loc wiring and external cable termination

We have all been there about the time constraint at the end of jobs. Signalling testing is always last on the planned work before handback and inevitably this puts pressure on the signalling staff reconnecting and testing all the signalling assets. The handback time generally does not move when previous work delay happens, this is where you need strong willed signalling testers to actually state they will be late because they still need to carry out the testing properly. Afterall, if this incident resulted in a crash, it will be the signalling tester who is held accountable and possibly up in the dock not the PICOP or control or the signaller

 

[Surreytraveller]

Why was the signal in rear not displaying a yellow aspect? If it was only the wires connected to this signal that were connected up wrong, surely the signal in rear was connected up correctly?
Or do the same wires feed the relay for the previous signal too?

 

[68000]

Why was the signal in rear not displaying a yellow aspect? If it was only the wires connected to this signal that were connected up wrong, surely the signal in rear was connected up correctly?
Or do the same wires feed the relay for the previous signal too?

Edit: In this fault condition for 5C23, the signal in rear would have displayed Green as the wiring error is restricted to the aspect circuits of DY586.

 

[Tomnick]

View attachment 145307

In this fault condition, the signal in rear would have displayed Green but should have displayed Yellow. The driver in rear has no way of knowing if the signal should have displayed a Green or Yellow.

The signal in rear (DY584) was correctly displaying Yellow?

 

[Surreytraveller]

View attachment 145307

In this fault condition, the signal in rear would have displayed Green but should have displayed Yellow. The driver in rear has no way of knowing if the signal should have displayed a Green or Yellow.

So it was two signals affected, not just one

 

[GC class B1]

View attachment 145307

In this fault condition, the signal in rear would have displayed Green but should have displayed Yellow. The driver in rear has no way of knowing if the signal should have displayed a Green or Yellow.

From my reading of the chart above it was only the signal aspect that was incorrect. I also thought the signal in the rear would be showing a yellow.

 

[Surreytraveller]

From my reading of the chart above it was only the signal aspect that was incorrect. I also thought the signal in the rear would be showing a yellow.

The report doesn't mention anything about the previous signal. If that was displaying yellow, then 1F02 would have been expecting to stop at the faulty signal anyway (even though it was displaying yellow vice red). But the fault must have followed through to the previous signal too

 

[68000]

DY584 correctly displayed a Yellow to 1F02 as 5C23 had left the overlap of DY586 and was in the signal section ahead therefore DY586 should have displayed a Red (but was displaying a Yellow). The report does not give any detail of the circuitry for DY584 and I assume the DY586 circuitry is also cut into DY584 circuits using spare contacts of DY586 relays

== Doublepost prevention - post automatically merged: 23 Oct 2023 ==

From my reading of the chart above it was only the signal aspect that was incorrect. I also thought the signal in the rear would be showing a yellow.

You are right, bad wording from me. Really depends which train is the subject. The fact 5C23 encountered a Red (vice Yellow), the expectation is that the signal in rear should have displayed Yellow but the fault was restricted to the signal aspect wiring therefore the interlocking sees the bigger picture and 5C23 encountered a Green on DY584

== Doublepost prevention - post automatically merged: 23 Oct 2023 ==

The signal in rear (DY584) was correctly displaying Yellow?

Yellow For 1F02 and Green for 5C23

 

[theageofthetra]

I'm just surprised that the RAIB is not as concerned as you rightly are. It's somewhat worrying that the RAIB doesn't appear to see the need for change.

100 % agree, I find it incredible that at no point a REC call occurred.

 

[Annetts key]

I was an SMTH tester over 20 years ago and almost all signalling technicians and testers use head torches for location case lighting for work of this nature. I must say I am surprised that certified signal technicians or testers did not know the difference between internal loc wiring and external cable termination

I’m lucky in that most of the location cupboards that I’ve worked in, at least had a 35W filament lamp or the more modern type of lamp (although the position of some of these means they are not as useful as they could be). However, I never rely on just a head torch even if there is a loc light for planned installation or testing work. I always take a portable battery powered flood light or hand torch as well. Where possible, actually two flood lights or hand torches so I can illuminate from two different directions (either side of me).

Why was the signal in rear not displaying a yellow aspect? If it was only the wires connected to this signal that were connected up wrong, surely the signal in rear was connected up correctly?
Or do the same wires feed the relay for the previous signal too?

The signal in rear (DY584) was correctly displaying Yellow?

So it was two signals affected, not just one

Okay, this is obviously causing confusion. It’s the interlocking that decides on what aspect each signal should be showing. For the faulty signal, the interlocking is of the relay type.

In this incident, the interlocking was working as designed, that is, it was working correctly. The problem was the wiring between the interlocking (the control relays) and the signal head.

The faulty signal, DY586 was not showing the correct aspect as commanded by the interlocking. The signal in rear, DY584 was correctly showing the aspect as commanded by the interlocking.

The interlocking has no way of knowing what colour light the signal it controls is actually producing.

The result is that when DY586 was supposed to be showing a yellow, because of the wiring fault it actually showed a red. The interlocking was commanding it to show a yellow. Hence the signal in rear, DY584 was commanded to show a green.

To a driver, this is an incorrect aspect sequence. They had no yellow to warn them of a red. However, with this particular fault, the section between DY586 to the next signal was clear and safe for the train.

When the interlocking commanded DY586 to show a red, because of the fault, it actually showed a single yellow. The interlocking for DY584 commanded that signal to show a single yellow. DY586 showing a yellow instead of a red is a unprotected wrong side failure, the worst category of signalling failure.

Note that with this fault, signal DY584 was correctly telling drivers the limit of authority of how far the section was clear ahead. SIgnal DY586 was not doing this. So with a train in the section ahead of DY584, it showed a red. With a train between BY586 and the following signal, DY584 showed a single yellow.

 

[LAX54]

I'm sorry but you're talking rubbish. The railway does not work on line of sight - the driver will not be expecting to stop at the next signal if they pass a green. If that were the case, trains would have to crawl around the network.

Looks like 2HAP does not have much idea on how the Railway and Signalling works.

 

[Annetts key]

One thing that I did spot, but which is not really mentioned in the report, is the action that should occur if there is a requirement to remove a wire that is not labelled.

In an earlier version of the SMTH document, test plan CA01 would have been used to disconnect a cable or wire.

Here’s the beginning of that test plan:

Test Plan CA01 (Remove and Refit a Cable Core or Wire) issue 07 dated 07/03/2020

Includes:
All cable core/cores or a single wire including line wires where the removal and refitting of the same correctly labelled cable core/wire or line wire is to be carried out.
A previously removed or broken, but correctly labelled, cable core(s)/wire including line wires, which is causing a fault (proceed from step 06).

Excludes:
Refitting any removed or broken cable core(s)/wires including line wires, not correctly labelled.

Then the following note:

Where a previously removed or broken, or not correctly labelled cable core(s)/wire including line wires, is encountered and is causing a fault, see NR/SMTH/Part03/A06 (Defined Check: check for correct labelling).

Followed by step 1:

1. CORRELATION CHECK cable core/wire to the wiring diagram (where more than one cable core/wire/line wire is to be removed at once).

The trainers that I had were very clear, if removing more than one wire, all unlabelled wires should be fitted with permanent or temporary labels.

In the same version of SMTH it says in part A:

Test/A06
Defined Check: Check for Correct Labelling
Issue No: 06 Issue Date: 04/09/2021

The relevant bit for wires and cables being:

3. Correct Labelling of Cables and Wires
Many Maintenance Test Plans specify the correct labelling and subsequent independent checking of cables and wires where disconnection is required as part of the like-for-like replacement or for the temporary diversion of circuits.

A wire or cable core is correctly labelled when it is uniquely marked or identified, and its label corresponds to the diagram. This can be achieved by checking the wire or cable core is terminated correctly prior to labelling or, where it is not terminated, its identity shall be physically traced or electrically proved. Electrical proving shall require positive proof of connection rather than apparent absence of a connection. Use of the CONTINUITY TEST might be required. If practical, wires shall be removed and refitted one at a time.

Cable core numbers or wire bead markers may be acceptable when clearly visible and unambiguous. In other cases, a tie-on or stick-on temporary label might be required. Temporary labels shall not damage either wires or equipment. They shall be removed on completion of the work or on rectification of a temporary diversion to avoid confusion. The essential feature is that everyone likely to be involved in the work or testing shall understand the labelling system adopted.

A process for the temporary labelling of diverted cable cores is given in
NR/SMTH/Part01/Module/12 (The Diversion of a Circuit/Relay Contact or Emergency Equipment Relocation).
END

Correlation Check:

Check A07
Defined Check: Correlation Check
Issue No. 05 Issue Date: 05/03/11

1. Purpose
Correlation is the comparison of an existing signalling system with the current design records, to check that the two are in agreement.

This involves:

Physical tracing or electrical proving of circuitry/ cables to ensure that they run to where the diagrams state.

And:

Correlation requires two members of staff, one of whom shall be deemed competent to carry out correlation. The assistant shall check the equipment and the competent correlator shall mark the diagrams.
The equipment shall be checked TO THE DIAGRAMS to avoid any chance of differences to the diagram being missed.

Marking the diagram only being relevant if you are using a copy, the maintenance copy should bot be marked.

For the Correlation Check, there are two versions, the full version would not be needed here, the SMTH Correlation version would be sufficient:

SMTH Correlation
1. Check that the correct circuit is identified for correlation.

2. Check that a diagram is available to correlate the circuit against. Note that very simple correlation can be made against the maintenance copy of the drawings. Should the work be more complicated, a photocopy of the relevant drawing is acceptable? (Note that where a photocopy is used, the copy should be kept with the NR/SMTH test record on completion of testing).
3. If a photocopy is used, Check that the photocopy and the maintenance copy are the same issue.

However, please note that the current version of the SMTH document is rather different.

Test plan CA12 does not include a Correlation Check. It does have step 2 and step 3:

2. Check cable is correctly labelled.

This means that the actual multicore cable has a label fitted to it with its name or number, it’s not referring to an individual wire/cable core.

3. Check cable cores are correctly labelled.

But there is no note to say how to proceed if the labelling is not correct.

The after re-connection section says:

7. Check cable cores are replaced as labelled.

Then the last step is:

13. Check or arrange for correct labelling of the cable or cable cores.

My thoughts are that the SMTH documentation really needs to be reviewed as over the years with all the alterations, it’s become inconsistent and in some areas, it’s not as clear as it should be.

At no point should safety critical signalling work involve having multiple unlabelled wires/cable cores being disconnected where they are intended to be reconnected at a later stage. Or just as bad, where the labelling is not readable or where the existing labelling is not sufficiently unique for correctly identifying individual wires/cable cores (relevant because sometimes multiple cables are terminated on the same cable terminal strip).

If a technician does find that they have multiple unlabelled wires that need reconnecting, tracing the wires as described in the Correlation Check should be done.

 

[DerekC]

Thanks. The important thing here though is that there's no Rule Book requirement for 1F02 to be advised at all, nor - in those first few minutes of the incident - any realisation that there was anything amiss with the signalling system that'd justify advising 1F02. It was only the initial assumption that it was a Cat A SPAD (in old money) combined with the local knowledge of the signaller on the adjacent workstation that led to the Derby signaller doing it. We should be eternally glad that they did, because otherwise we'd almost certainly have been looking at a collision, but I don't know how you could ever make it an absolute requirement to do anything like that.

Based on what @baz962 says, the worry is that RAIB doesn't seem to have picked up this issue at all so didn't consider whether or not the existing rules should or can be amended. It's their job to identify learning points and areas where change may be needed and they seem to have missed one here. I think this is sufficiently serious that one of the people with first hand knowledge - ideally the driver of 1F02 - should raise the concern formally. I would have thought the route for this would be via EMR, but I am not sure how it works these days.

 

[baz962]

Based on what @baz962 says, the worry is that RAIB doesn't seem to have picked up this issue at all so didn't consider whether or not the existing rules should or can be amended. It's their job to identify learning points and areas where change may be needed and they seem to have missed one here. I think this is sufficiently serious that one of the people with first hand knowledge - ideally the driver of 1F02 - should raise the concern formally. I would have thought the route for this would be via EMR, but I am not sure how it works these days.

To be fair the driver didn't realise the actual extent until he got back to London. But the feeling in our messroom was that the next train was potentially a collision. People here saying that the signaller just thought it was just maybe low adhesion and a spad . Ok if you have had one train report a potential failure I would understand. But you have had the first train go through a red and then the second train that you cautioned go through a yellow that should of been red and now have two trains in a section and still don't caution the next train coming through faster than the second train that you did caution. Makes no sense to me and my colleagues. Just an edit after re reading. I'm not on about 1F02 which was the second train that ended up in the section. I am referring to 1F01 which was approaching the two trains already in one section and could potentially have been number three in the section.

 

[GC class B1]

Based on what @baz962 says, the worry is that RAIB doesn't seem to have picked up this issue at all so didn't consider whether or not the existing rules should or can be amended. It's their job to identify learning points and areas where change may be needed and they seem to have missed one here. I think this is sufficiently serious that one of the people with first hand knowledge - ideally the driver of 1F02 - should raise the concern formally. I would have thought the route for this would be via EMR, but I am not sure how it works these days.

I fully support this view as I have also identified areas of concern in previous RAIB investigations that have not been dealt with in the reports.

 

[Re 4/4]

If the driver of 1F02 sees a train in the section after passing a yellow signal, should that be grounds for them to hit the Red Button, irrespective of what the signaller is doing?

I'm not criticising the driver who no doubt had enough on his mind at the time, I'm more asking if it might be prudent in a future rule book to say that a case like this is a valid reason for stopping everything in the area?

 

[Surreytraveller]

100 % agree, I find it incredible that at no point a REC call occurred.

For a REC call, someone has to immediately realise that something is seriously wrong. The only person who would realise that is the driver of 1F02. The driver of 5C23 is probably racking their brains thinking surely they've missed a yellow, and the Signaller, unless their eyes are glued to the panel, is just thinking 5C23 has passed a red because of the train in front, not realising that the train in front is a section further in front than the signal displays to the driver.
Even the driver of 1F02 is probably under the impression the situation is under control owing to the cautioning they received (even if it wasn't an official caution).

 

[Nicholas Lewis]

I would suggest Para 86 really is nub of it

There were four opportunities for the wiring problem with the signal to have been identified by the test steps in test B07

RAIB seeks to find excuses for why this didn't happen and whilst im sure time pressures did play a part was surprised to read that people are still allowing themselves to be pressured and HOTR works manager are exerting pressure on them. These teams are doing this night in night out for over ten years so should be well versed on not putting undue pressure on testing staff. Easy to use as an excuse in my opinion. The bottom line is SMTH process wasn't followed due to suspect competency from a contracting organisation that is left to self assure. NR really needs to consider whether such safety critical activities should be left to external organisation here and that final sign off should always be from NR employee even if contractors are used.

 

[Signal Head]

I would suggest Para 86 really is nub of it



RAIB seeks to find excuses for why this didn't happen and whilst im sure time pressures did play a part was surprised to read that people are still allowing themselves to be pressured and HOTR works manager are exerting pressure on them. These teams are doing this night in night out for over ten years so should be well versed on not putting undue pressure on testing staff. Easy to use as an excuse in my opinion. The bottom line is SMTH process wasn't followed due to suspect competency from a contracting organisation that is left to self assure. NR really needs to consider whether such safety critical activities should be left to external organisation here and that final sign off should always be from NR employee even if contractors are used.

Network Rail are moving more towards self-assurance, no doubt under pressure to cut costs in the run up to the conversion to GBR, which has seen lots of experienced folk 'take the money and run' (and who can blame them?).

The railway seems to be running out of skills which were taken for granted across the 'S&T' function under BR, particularly on older mechanical and electro-mechanical kit and systems, and judging by this incident, on relay-based installations as well.

The self-assurance will extend to design too, and considering some of the howlers I'm aware of, that's worrying. Not wishing to overstate things, but from certain viewpoints, the industry seems to be full of lots of people who joined post-privatisation, who never got BR training, who have never been taught the nuances of signalling "dos and don'ts", and of course there are now commercial pressures - witness the concerns over the cost of overrunning the possession in this case.

Bad design puts even more pressure on testing staff who are the last line of defence in catching wrong side failures, and considering that many (most?) of the senior experienced testers from BR days have retired, or will do so in the relatively near future, I wouldn't be at all surprised to see more incidents of this nature.

 

[John Webb]

For a REC call, someone has to immediately realise that something is seriously wrong. The only person who would realise that is the driver of 1F02. The driver of 5C23 is probably racking their brains thinking surely they've missed a yellow, and the Signaller, unless their eyes are glued to the panel, is just thinking 5C23 has passed a red because of the train in front, not realising that the train in front is a section further in front than the signal displays to the driver.
Even the driver of 1F02 is probably under the impression the situation is under control owing to the cautioning they received (even if it wasn't an official caution).

The panel could not display the aspect of DY586 as set by the system. So even with eyes 'glued to the panel' the signaller was not given any clue that it wasn't showing what the system was telling it to show. Hence they were unable to appreciate there was a contradiction between what the system was telling the signal to do and what the driver of 5C23 was reporting.
Do more modern systems tell the signallers the aspects of all signals, even the fully automatic ones?

 

[Signal Head]

The panel could not display the aspect of DY586 as set by the system. So even with eyes 'glued to the panel' the signaller was not given any clue that it wasn't showing what the system was telling it to show. Hence they were unable to appreciate there was a contradiction between what the system was telling the signal to do and what the driver of 5C23 was reporting.
Do more modern systems tell the signallers the aspects of all signals, even the fully automatic ones?

It's possibly worth noting that the 'panel' (actually a VDU), even if it had indications, would only report the aspect based on the state of the controlling relays, since the wiring cross was between the relays and the aspect. This is different to Clapham, where the control relay (HR) was falsely energised.

 

[Gaelan]

It's possibly worth noting that the 'panel' (actually a VDU), even if it had indications, would only report the aspect based on the state of the controlling relays, since the wiring cross was between the relays and the aspect. This is different to Clapham, where the control relay (HR) was falsely energised.

Might've let them catch the problem faster, though - the signaler could've notice the signal was yellow despite the driver reporting a SPAD.

 

[Island 168]

As with the Clapham accident of 1988 this incident at Wingfield highlights the need for the actual signal aspect to be comfirmed to the signaller, not just that a particular wire has been energised.
One of the recommendations following the inquiry into the Abbots Ripton accident in 1876 was that signal aspects should be confirmed '...that they provide an indication to the signalman if they are not operating properly'. This sensible recommendation should surely apply to all signals whether semaphore or colour light signals.

 

[Gaelan]

As with the Clapham accident of 1988 this incident at Wingfield highlights the need for the actual signal aspect to be comfirmed to the signaller, not just that a particular wire has been energised.
One of the recommendations following the inquiry into the Abbots Ripton accident in 1876 was that signal aspects should be confirmed '...that they provide an indication to the signalman if they are not operating properly'. This sensible recommendation should surely apply to all signals whether semaphore or colour light signals.

I don't see how this would be practical, short of every signal having a camera pointed at the lights.

 

[foggy69]

If the driver of 1F02 sees a train in the section after passing a yellow signal, should that be grounds for them to hit the Red Button, irrespective of what the signaller is doing?

I'm not criticising the driver who no doubt had enough on his mind at the time, I'm more asking if it might be prudent in a future rule book to say that a case like this is a valid reason for stopping everything in the area?

I expect that once the driver of 1F02 had bought their train to a stand, in their mind the immediate emergency was over, there was no risk to other trains. A priority call to the box was all that was needed and for the box to protect the trains as necessary.

I don't remember reading in the report as to exactly when the wrongside failure issue was finally realised by the box, but hopefully it was soon after the call from 1F02, and at that point all trains in the area should have been stopped.

 

[baz962]

I expect that once the driver of 1F02 had bought their train to a stand, in their mind the immediate emergency was over, there was no risk to other trains. A priority call to the box was all that was needed and for the box to protect the trains as necessary.

I don't remember reading in the report as to exactly when the wrongside failure issue was finally realised by the box, but hopefully it was soon after the call from 1F02, and at that point all trains in the area should have been stopped.

And that is my point. 5c23 had the spad ok fair enough that is all the signaller knew at that point. But then 1F02 ended up passing a false yellow and in the same section and so now there were two trains in the same section and 1F01 was three or four minutes away doing 100 mph and the signaller didn't stop that train , which was about to be number three in that section.

 

[edwin_m]

The panel could not display the aspect of DY586 as set by the system. So even with eyes 'glued to the panel' the signaller was not given any clue that it wasn't showing what the system was telling it to show. Hence they were unable to appreciate there was a contradiction between what the system was telling the signal to do and what the driver of 5C23 was reporting.
Do more modern systems tell the signallers the aspects of all signals, even the fully automatic ones?

With relay interlockings as at Wingfield, the logic that works out what aspect to display is housed in a trackside cabinet near the signal. Any control from or indication to the panel would require an extra wire (conceptually, in practice it would go through various multiplex systems). For automatic signals in these areas the signaller only sees the track circuit occupation and can only replace selected signal to danger, the others having a trackside replacement switch as mentioned in the report. Even with controlled signals the signaller can only tell if they are red or not (shown on the panel as green), not the actual aspect displayed.

Solid State Interlocking, in use from about 1990 onwards, and I presume later Computer Based Interlockings, the aspect is worked out by software logic within the interlocking itself and transmitted via the data feed to the display, so the signaller sees what the interlocking is commanding the signal to display, unless the lamp proving circuit is broken in which case it is displayed as blank. There are various hybrid systems where the situation can be somewhere in between. In all cases there is still the risk of a wiring fault similar to the one here, physically connecting the wrong lamp to the trackside output.

I don't see how this would be practical, short of every signal having a camera pointed at the lights.

Or perhaps a colour-specific light detector within the signal head. But ultimately a signalling system is always going to have to rely on best practice in wiring, checking and testing, although the computer-based systems pass some of that responsibility to the data preparers.

 

[Tomnick]

And that is my point. 5c23 had the spad ok fair enough that is all the signaller knew at that point. But then 1F02 ended up passing a false yellow and in the same section and so now there were two trains in the same section and 1F01 was three or four minutes away doing 100 mph and the signaller didn't stop that train , which was about to be number three in that section.

I do get what you're saying – but how far do you go in the rules to mitigate a situation that shouldn't ever exist in the first place? I have to say that I'd probably have been inclined to make an emergency group call if I'd been the second driver, but that's with the benefit of experience in the signalling grade and probably more of an interest than most in the inner workings of the signalling system and the finer details of what led to the accident at Clapham Junction. Up until that point, though, I don't think it was reasonable to expect anyone to realise that anything was seriously amiss – it had all the hallmarks of a straightforward right-side signal reversion.

However, it's important to point out that 1F01, even if the "contact signaller" message hadn't been sent, would've correctly received a single yellow at DY584 and thus wouldn't be approaching the scene at linespeed – it doesn't lessen the severity of the errors made, but it does at least reduce the consequences in this specific situation.