Elizabeth line reliability issues

[rower40]

Unfortunately, when the problem is interference on both datalinks, it is very difficult to track down the cause. A process of elimination has to be done, with the techs visiting every single location cupboard where the datalinks visit.

One wonders if the second failure (Wednesday night) might not have been caused by some unintended disturbance of previously-working cables during the search for the original fault. Especially if the process to check the datalinks involves unplugging a connector, checking for bent pins or not-properly-seated plugs, then plugging it back in again.

 

[NSEWonderer]

Quite a few driver displacements and shortages especially noticeable on the WWC branch at the moment as with these things. Expecting it to slowly recover tho before next peak, delays wise.

 

[Goldfish62]

1129 from Reading this morning shown as on time on the screens at Reading and as announced. At the alloted time the train simply vanished from the screens. It didn't run, but no announcement or anything.

GWR also still in complete chaos this morning.

 

[Taunton]

Well it seems through running resumed to an extent somewhat around 1100 today (Thursday). Though when I was at Paddington at that time it all seemed quite confused, messages to go up to the mainline, nothing local westbound on the mainline boards, etc.

But down below there was a mystery train in the platform. "Stand Clear Of The Platform Edge, This Train Does Not Stop Here" on both indicators and constantly booming out on the PA. Except it did, with doors open, and was already full so must have been there for some time, with the driver frequently announcing it was for Abbey Wood. This charade was repeated at each station. Quite apart from it being ludicrous to make such announcements at stations with platform edge doors, passengers on each platform were shouting in through the doors where it was going, and the passengers were shouting back much more accurate information than the £ multi-million systems were capable of.

When I got to Canary Wharf a train ran in on the opposite side labelled for Reading, but RTT shows nothing for it after Paddington, so no idea if it went through or not.

 

[apbj]

At the alloted time the train simply vanished from the screens. It didn't run, but no announcement or anything.

This happens all the time, especially on the wizzy horizontal monitors above the platform doors. During disruption they might as well show old sitcoms because the running information is so wrong it's an active hindrance to customers. It's particularly annoying as these are what the platform staff (if they can be bothered) use for announcements to passengers.

It seems to be that the core/MTR use an information system that 'talks to' other TfL systems and displays but is't based on the same methods or is not as reliable for whatever reason. Again, the complexity (perhaps necessary, perhaps not) is causing problems for the Elizabeth line compared to the Tube/Overground.

 

[Horizon22]

This happens all the time, especially on the wizzy horizontal monitors above the platform doors. During disruption they might as well show old sitcoms because the running information is so wrong it's an active hindrance to customers. It's particularly annoying as these are what the platform staff (if they can be bothered) use for announcements to passengers.

It seems to be that the core/MTR use an information system that 'talks to' other TfL systems and displays but is't based on the same methods or is not as reliable for whatever reason. Again, the complexity (perhaps necessary, perhaps not) is causing problems for the Elizabeth line compared to the Tube/Overground.

But this was at Reading, so that’s a GWR/ classically managed system.

But yes as with other displays generally with intensive metro services, cancellations are just removed from the platform screen doors CIS information once input; they don’t show “cancelled”. Even on regular NR infrastructure, if a train is delayed past departure time and then cancelled, it is often only left showing for a few minutes, if at all.

 

[Taunton]

I do get the feeling that, after 15 months of Elizabeth service, they have just got comfortable with the information systems they have got, and the fact that those on the Core show rubbish, in particular when the service has been disrupted, as this week, or is otherwise not running in accordance with a pre-loaded timetable, is just now accepted as 'one of those things'.

 

[Horizon22]

I do get the feeling that, after 15 months of Elizabeth service, they have just got comfortable with the information systems they have got, and the fact that those on the Core show rubbish, in particular when the service has been disrupted, as this week, or is otherwise not running in accordance with a pre-loaded timetable, is just now accepted as 'one of those things'.

I’ve seen many other operators display “rubbish” during service disruption, including London Underground. I hardly think this is unique (unfortunately).

Certainly some different quirks and issues (like passenger trains showing “not in service”) that continue to be a problem but actually this seems to be happening less than it was.

 

[Annetts key]

I’m no longer involved (and my involvement was for a very short amount of time many years ago) with the Customer Information Systems (CIS).

There are a number of problems. First, and most significantly, due to the increased automation, there are less real human beings to manually update/change or check the CIS information. Especially when the train service goes pear shaped for any reason.

The second problem, is that during a failure of the signalling system, because the CIS ultimately uses train movement information from the Network Rail Train Describer (TD) system (for Network Rail infrastructure), and the TD gets its train movement information from the signalling system, it will end up displaying incorrect information.

Of course, the above points are not the only issues.

 

[Horizon22]

Some more fun this evening with another signalling fault, this time near Southall.

 

[800301]

Some more fun this evening with another signalling fault, this time near Southall.

Axle counter failure, Southall West, route locked into Southall Yard/Brentford on the Up Main, GWR standard contingency of Reading to Didcot and Reading to Newbury for 387 services

 

[ChewChewTrain]

I realise the “TS” word is probably akin to swearing around here, but let’s face it: for us, it’s quite fun looking at SignalMaps and trying to work out what’s going on, who’s being let through next, etc. As we pootle into Paddington, I’m somewhat less bored than most of the passengers around me look.

Silver linings.

 

[Bald Rick]

I do get the feeling that, after 15 months of Elizabeth service, they have just got comfortable with the information systems they have got, and the fact that those on the Core show rubbish, in particular when the service has been disrupted, as this week, or is otherwise not running in accordance with a pre-loaded timetable, is just now accepted as 'one of those things'.

I can confirm that this absolutely not the case.

 

[Goldfish62]

I can confirm that this absolutely not the case.

I agree. Behind the scenes there's a lot of sh®t flying around

 

[GordonT]

I agree. Behind the scenes there's a lot of sh®t flying around

Would have been a good week for filming a "Paddington Station" tv episode if that had still been on the go.

 

[Dave2307]

So Friday morning and the major EL hiatus of the past 3 days seem to have been resolved. However still disruption this morning due the ol
h so familiar ‘defective train’ at Paddington eastbound.

Itseems obvious this is not a rolling stock issue, it is the failure of the switchover of systems from driver to computer control for the run through the central section. This has happened at least 3 days per week since through running began. It is always hard to track down and eliminate intermittent issues but it has been 6 months+!

Also why does the same issue not arise at the eastern end of the central section, or are we in the west too consumed by our poor service to notice?

This is not going away!

 

[JustPassingBy]

Was the cause of the data link issue (that affected both the primary and backup systems) identified and fixed? Or has the problem gone away on it's own? (and so liable to recur).

 

[Taunton]

However still disruption this morning due the oh so familiar ‘defective train’ at Paddington eastbound.


It seems obvious this is not a rolling stock issue, it is the failure of the switchover of systems from driver to computer control for the run through the central section. This has happened at least 3 days per week since through running began. It is always hard to track down and eliminate intermittent issues but it has been 6 months+!

Also why does the same issue not arise at the eastern end of the central section, or are we in the west too consumed by our poor service to notice?

This is not going away!

Unfortunately it's again right at 0700 in the morning peak, with the usual array it appears of inbound trains either sat along the eastbound GWML for half an hour, or diverted to the mainline station, with the concomitant subsequent cancellations. All those morning arrival air passengers at Heathrow. What a welcome to Britain!

I do suspect the four year late start has meant that the skills of those who designed and developed all the complex aspects have dissipated. Quite a proportion of the seniors whose ideas are in all this will have retired.

 

[Bald Rick]

it is the failure of the switchover of systems from driver to computer control for the run through the central section.

And that system is on the train. Hence rolling stock (although a little more complicated).

Was the cause of the data link issue (that affected both the primary and backup systems) identified and fixed?

Yes.

I do suspect the four year late start has meant that the skills of those who designed and developed all the complex aspects have dissipated. Quite a proportion of the seniors whose ideas are in all this will have retired.

Theres some of that, but those who built and commissioned it are still there.

 

[Dave2307]

And that system is on the train. Hence rolling stock (although a little more complicated).



Yes.


Theres some of that, but those who built and commissioned it are still there.

Yeah the system is on the train you are right but I was just trying to differentiate between the physical aspect of the train and the systems added to it. The fact the latter keep failing makes you think there is a fundamental problem buried deep which has still not been sorted. I think the other 1100 members of the Hanwell users group would agree.

 

[Horizon22]

Yeah the system is on the train you are right but I was just trying to differentiate between the physical aspect of the train and the systems added to it. The fact the latter keep failing makes you think there is a fundamental problem buried deep which has still not been sorted. I think the other 1100 members of the Hanwell users group would agree.

Ultimately it’s software related. And that software can be incredibly complex so I think you’d have to be a software engineer to explain it fully!

That being said the last few weeks is hugely improved on June which was was fairly atrocious for long sit downs and fleet failures.

 

[jayah]

As the railway reduces the number of staff and resources, should a failure like the above occur, it takes longer to locate the problem and to try to implement manual signalling. For example, it’s not uncommon for the staff on the ground to find that they don’t have enough point clips to secure all the points.

Lack of staff, staff running out of shift time, lack of spares all add to the problems.

One point discussed at length during the NR strikes was the practice of staff from other territories / regions assisting. Does that happen now e.g. techs from LNW or Wessex attending a major incident on GWR, or do they 'stay in their lane'? Perhaps they always did?

 

[Taunton]

That being said the last few weeks is hugely improved on June which was was fairly atrocious for long sit downs and fleet failures.

If this week just ending is seen as part of some 'huge improvement' then I'm staggered.

 

[Purple Train]

If this week just ending is seen as part of some 'huge improvement' then I'm staggered.

I think @Horizon22 meant that it is a huge improvement in on-train software reliability, rather than the reliability of the pre-existing signalling.

 

[Annetts key]

One point discussed at length during the NR strikes was the practice of staff from other territories / regions assisting. Does that happen now e.g. techs from LNW or Wessex attending a major incident on GWR, or do they 'stay in their lane'? Perhaps they always did?

Prior to Modernising Maintenance (the changes that Network Rail are introducing), maintenance staff from a different route/region could not be compelled to work on an adjacent route/region. Modernising Maintenance changes this, so in future, staff can be instructed to work “across the border”.

Staff could volunteer though.

In this incident, as far as I know (I’m not involved, so don’t have all the details) the S&T (Signal & Telecom) for this area (Western Route East Delivery Unit) were only assisted by a an external company (Park Signalling) and volunteers from Western Route Central Delivery Unit.

In any case, each team on track would need a COSS (Controller Of Site Safety - this is safety in relation to the risk from trains, not any other safety concerns) who knows the local area.

All staff working on or testing signalling equipment are also required to be fully trained and competent on the type of equipment they are working on. As the different routes/regions don’t all use the same type/make of equipment, this may or may not be a factor. With this failure, the problem was SSI (Solid State Interlocking) datalink cables and the SSI modules that use the datalink system. This is fairly standard across the different routes/regions.

 

[Horizon22]

I think @Horizon22 meant that it is a huge improvement in on-train software reliability, rather than the reliability of the pre-existing signalling.

Indeed.

 

[jayah]

Prior to Modernising Maintenance (the changes that Network Rail are introducing), maintenance staff from a different route/region could not be compelled to work on an adjacent route/region. Modernising Maintenance changes this, so in future, staff can be instructed to work “across the border”.

Staff could volunteer though.

In this incident, as far as I know (I’m not involved, so don’t have all the details) the S&T (Signal & Telecom) for this area (Western Route East Delivery Unit) were only assisted by a an external company (Park Signalling) and volunteers from Western Route Central Delivery Unit.

In any case, each team on track would need a COSS (Controller Of Site Safety - this is safety in relation to the risk from trains, not any other safety concerns) who knows the local area.

All staff working on or testing signalling equipment are also required to be fully trained and competent on the type of equipment they are working on. As the different routes/regions don’t all use the same type/make of equipment, this may or may not be a factor. With this failure, the problem was SSI (Solid State Interlocking) datalink cables and the SSI modules that use the datalink system. This is fairly standard across the different routes/regions.

Clearly a big benefit from Modernising Maintenance when this eventually happens, since it is evident from your postings that shortage of staff is a significant issue here. Inevitably a relatively small 'delivery unit' will be overwhelmed by something of this magnitude and the amount of time the incident was ongoing.

Competence should not and must not be allowed to become a blocker for this very sensible and much needed progression. Network Rail needs to be focused on the needs of the railway's customers, not its own IR / internal politics which have held sway for too long.

There is only limited value in getting other staff competent if they can refuse to go when the balloon goes up, and it sounds like anyone 'volunteering' against the wishes of the union risks become persona non-grata under current conditions.

Granted we don't know if the number of volunteers from the neighbouring delivery unit were significant or not, but that area is probably not even the third nearest to the incident.

 

[Goldfish62]

Yesterday I travelled from Whitechapel to Reading. Everything was like clockwork and as it should be. So it does work as advertised sometimes! All the trains in the central core were incredibly busy.

 

[SWT_USER]

Yesterday I travelled from Whitechapel to Reading. Everything was like clockwork and as it should be. So it does work as advertised sometimes! All the trains in the central core were incredibly busy.

To counter... Yesterday I traveled from T4 to Custom House. Heavily delayed on the way out resulting in fast running from Heathrow to Paddington, didn't even make it to T4 on the return as the driver announced at Hayes that the train would instead be terminating at West Drayton.

All platforms between Hayes and Ealing Broadway had lots of people on so it must have been very cosy on any services that did actually stop there.

 

[Acton1991]

To counter... Yesterday I traveled from T4 to Custom House. Heavily delayed on the way out resulting in fast running from Heathrow to Paddington

Why do they continue to do this? Surely there’s a better way of making back time. It’s not helpful for those of us waiting further down the line